2024-09-18T00:56:05.693 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T00:56:05.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T00:56:05.806 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909695' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7WO3oM2Q4yjhxBxKd/6iJY3/m826ifegLDPxFP9LlVmTRKiDtFyiFws/mBMg60ETh3sG68xKTVYZRxp0kuj/g= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwWsyTgolgu0Fbg0FAUr/4DIEhKvWbgbf7nRhUSrBCat7yDPz8h2c0oS98nZel8UVHLJ9lhhUv1ogmHetTgvTc= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 deep_copy: true ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T00:56:05.806 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-18T00:56:05.807 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-18T00:56:05.807 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T00:56:05.808 INFO:teuthology.task.internal:Checking packages... 2024-09-18T00:56:05.829 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-18T00:56:05.830 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T00:56:05.830 INFO:teuthology.packaging:ref: None 2024-09-18T00:56:05.830 INFO:teuthology.packaging:tag: None 2024-09-18T00:56:05.830 INFO:teuthology.packaging:branch: squid 2024-09-18T00:56:05.830 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:56:05.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-18T00:56:06.043 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-18T00:56:06.044 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T00:56:06.051 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T00:56:06.051 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T00:56:06.078 INFO:teuthology.task.internal:Saving configuration 2024-09-18T00:56:06.091 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T00:56:06.114 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T00:56:06.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:50:55.687161', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7WO3oM2Q4yjhxBxKd/6iJY3/m826ifegLDPxFP9LlVmTRKiDtFyiFws/mBMg60ETh3sG68xKTVYZRxp0kuj/g='} 2024-09-18T00:56:06.162 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:50:55.688329', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwWsyTgolgu0Fbg0FAUr/4DIEhKvWbgbf7nRhUSrBCat7yDPz8h2c0oS98nZel8UVHLJ9lhhUv1ogmHetTgvTc='} 2024-09-18T00:56:06.162 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T00:56:06.167 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T00:56:06.167 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T00:56:06.167 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T00:56:06.273 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb108717370>, signals=[15]) 2024-09-18T00:56:06.273 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T00:56:06.278 INFO:teuthology.task.internal:Opening connections... 2024-09-18T00:56:06.278 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T00:56:06.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:56:06.356 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T00:56:06.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:56:06.430 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T00:56:06.436 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-18T00:56:06.454 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-18T00:56:06.454 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-18T00:56:06.511 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T00:56:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T00:56:06.513 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-18T00:56:06.537 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-09-18T00:56:06.556 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-09-18T00:56:06.556 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-09-18T00:56:06.613 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-09-18T00:56:06.613 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-09-18T00:56:06.613 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T00:56:06.614 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T00:56:06.615 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-09-18T00:56:06.646 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T00:56:06.655 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T00:56:06.693 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T00:56:06.694 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:56:06.696 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:56:06.712 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T00:56:06.720 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T00:56:06.721 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:56:06.754 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:56:06.861 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T00:56:06.915 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T00:56:06.929 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T00:56:06.929 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T00:56:06.930 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:56:06.930 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:56:06.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:06.947 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-18T00:56:06.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:06.948 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-18T00:56:07.005 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-503.el9.x86_64 2024-09-18T00:56:07.006 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-503.el9.x86_64 2024-09-18T00:56:07.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-09-18T00:56:07.007 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-503.el9.x86_64 2024-09-18T00:56:07.007 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.14.0-503.el9.x86_64 2024-09-18T00:56:07.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum install -y kernel 2024-09-18T00:56:09.301 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-18T00:56:10.409 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.3 MB 00:01 2024-09-18T00:56:11.498 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-18T00:56:15.401 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 20 MB 00:03 2024-09-18T00:56:17.095 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-18T00:56:19.312 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-18T00:56:20.743 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-18T00:56:21.316 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 12 MB/s | 23 MB 00:01 2024-09-18T00:56:22.185 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-18T00:56:23.972 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-18T00:56:26.822 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-18T00:56:28.722 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T00:56:28.722 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T00:56:28.821 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T00:56:28.829 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-09-18T00:56:28.830 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:56:28.831 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 79 M 2024-09-18T00:56:28.831 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 126 M 2024-09-18T00:56:28.831 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T00:56:29.065 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 321 kB/s | 36 kB 00:00 2024-09-18T00:56:29.352 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-18T00:56:29.707 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-18T00:56:30.266 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-18T00:56:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-18T00:56:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:56:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-18T00:56:31.265 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T00:56:31.292 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T00:56:31.292 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T00:56:31.378 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T00:56:31.390 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:56:31.391 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 79 M 2024-09-18T00:56:31.392 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 126 M 2024-09-18T00:56:31.392 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T00:56:31.614 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T00:56:31.614 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T00:56:31.978 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 65 kB/s | 36 kB 00:00 2024-09-18T00:56:32.572 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T00:56:32.572 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T00:56:33.845 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.5 MB/s | 16 MB 00:02 2024-09-18T00:56:34.486 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-18T00:56:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T00:56:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 9.2 MB/s | 35 MB 00:03 2024-09-18T00:56:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:56:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:Total 21 MB/s | 79 MB 00:03 2024-09-18T00:56:35.628 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:56:35.719 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:56:35.862 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T00:56:36.228 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T00:56:36.229 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T00:56:37.193 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T00:56:37.194 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T00:56:38.458 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:56:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:56:39.659 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T00:56:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:56:40.403 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:56:43.192 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:56:43.329 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:56:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:56:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:56:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:56:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:56:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:56:53.079 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:26.480 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:26.519 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:26.794 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:26.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:57:26.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:57:26.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T00:57:27.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:57:27.606 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T00:57:27.940 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:57:28.032 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:28.032 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:57:28.032 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T00:57:28.032 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:57:28.032 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:57:28.033 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T00:57:28.033 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:57:28.033 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:57:28.091 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:28.092 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:57:28.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 82.el9 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T00:57:28.126 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8274670 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T00:57:28.127 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T00:57:28.128 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-09-18T00:57:28.128 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T00:57:28.128 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T00:57:28.129 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T00:57:28.129 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-508.el9.x86_64 2024-09-18T00:57:28.129 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T00:57:28.761 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-09-18T00:57:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T00:57:30.098 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-09-18T00:57:30.101 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-18T00:57:30.116 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.AiGHelEKpf 2024-09-18T00:57:30.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AiGHelEKpf 2024-09-18T00:57:30.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.AiGHelEKpf 2024-09-18T00:57:30.327 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.AiGHelEKpf is 6KB 2024-09-18T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.AiGHelEKpf 2024-09-18T00:57:30.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T00:57:30.461 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T00:57:30.461 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T00:57:30.461 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T00:57:30.461 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T00:57:30.464 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T00:57:30.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-09-18T00:57:31.281 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:31.321 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:31.687 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:31.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:57:31.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:57:31.687 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:57:32.559 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T00:57:32.923 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:57:33.016 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:33.016 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:57:33.016 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T00:57:33.016 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:57:33.016 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:57:33.016 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T00:57:33.016 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:57:33.017 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:57:33.075 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:57:33.076 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:57:33.076 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2024-09-18T00:57:33.108 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 82.el9 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8274670 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T00:57:33.109 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T00:57:33.110 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T00:57:33.111 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T00:57:33.111 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 5.14.0-508.el9.x86_64 2024-09-18T00:57:33.111 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T00:57:33.783 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-09-18T00:57:35.060 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T00:57:35.077 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-09-18T00:57:35.079 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-18T00:57:35.095 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.Gu83NptLRa 2024-09-18T00:57:35.095 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Gu83NptLRa 2024-09-18T00:57:35.163 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.Gu83NptLRa 2024-09-18T00:57:35.312 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.Gu83NptLRa is 6KB 2024-09-18T00:57:35.363 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.Gu83NptLRa 2024-09-18T00:57:35.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T00:57:35.442 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T00:57:35.442 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T00:57:35.442 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T00:57:35.443 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T00:57:35.444 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T00:57:35.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-09-18T00:58:00.637 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T00:58:00.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T00:58:00.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:58:05.603 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T00:58:05.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T00:58:05.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:58:19.069 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-18T00:58:23.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-18T00:58:28.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T00:58:28.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:58:31.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-18T00:58:32.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T00:58:32.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:58:35.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-18T00:58:43.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T00:58:43.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:58:48.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T00:58:48.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:59:01.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-18T00:59:16.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T00:59:16.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:59:16.816 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-18T00:59:17.442 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T00:59:17.442 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T00:59:17.442 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-18T00:59:17.502 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-508.el9.x86_64 2024-09-18T00:59:17.503 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T00:59:17.503 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:59:17.503 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T00:59:18.504 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T00:59:18.505 DEBUG:teuthology.orchestra.run.smithi120:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:59:18.871 INFO:teuthology.orchestra.run.smithi120.stdout:ttyS1 2024-09-18T00:59:18.898 DEBUG:teuthology.parallel:result is None 2024-09-18T00:59:48.006 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T01:00:03.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T01:00:03.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T01:00:03.342 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-18T01:00:03.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T01:00:03.729 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-18T01:00:03.729 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-18T01:00:03.744 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-508.el9.x86_64 2024-09-18T01:00:03.744 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T01:00:03.744 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T01:00:03.744 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T01:00:04.745 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T01:00:04.746 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T01:00:04.831 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-09-18T01:00:04.851 DEBUG:teuthology.parallel:result is None 2024-09-18T01:00:04.851 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T01:00:04.860 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T01:00:04.860 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T01:00:04.863 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T01:00:04.881 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T01:00:04.909 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T01:00:04.959 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T01:00:04.959 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T01:00:04.961 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T01:00:05.003 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T01:00:05.010 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T01:00:05.010 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T01:00:05.041 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T01:00:05.075 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T01:00:05.080 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T01:00:05.088 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T01:00:05.091 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T01:00:05.093 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T01:00:05.100 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T01:00:05.100 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T01:00:05.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T01:00:05.161 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T01:00:05.169 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T01:00:05.170 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T01:00:05.200 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T01:00:05.217 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T01:00:05.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T01:00:05.346 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T01:00:05.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T01:00:05.469 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:00:05.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T01:00:05.535 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T01:00:05.581 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T01:00:05.631 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T01:00:05.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T01:00:05.759 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:00:05.759 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T01:00:05.826 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-18T01:00:05.828 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-09-18T01:00:05.876 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:00:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:00:06.370 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T01:00:06.378 INFO:teuthology.task.internal:Starting timer... 2024-09-18T01:00:06.378 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T01:00:06.438 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T01:00:06.467 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-18T01:00:06.468 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-18T01:00:06.528 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-18T01:00:06.530 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-09-18T01:00:06.576 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-09-18T01:00:06.578 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T01:00:06.578 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-18T01:00:06.599 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-18T01:00:06.600 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-09-18T01:00:06.628 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-09-18T01:00:06.629 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-09-18T01:00:06.629 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:00:06.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:00:06.667 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:00:06.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:00:06.695 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T01:00:06.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-18T01:00:06.740 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-09-18T01:00:06.773 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T01:00:06.783 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-18T01:00:06.786 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-18T01:00:06.899 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T01:00:06.909 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T01:00:06.910 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi120.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T01:03:54.141 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-09-18T01:03:54.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T01:03:54.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T01:03:54.222 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-18T01:03:54.263 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-18T01:03:54.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T01:03:54.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T01:03:54.339 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-18T01:03:54.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-18T01:03:54.417 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T01:03:54.428 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T01:03:54.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T01:03:54.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:03:54.432 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T01:03:54.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:03:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T01:03:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T01:03:54.487 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T01:03:54.498 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-18T01:03:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T01:03:54.508 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-18T01:03:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T01:03:54.526 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-09-18T01:03:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T01:03:54.538 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-09-18T01:03:54.552 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T01:03:54.566 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T01:03:54.592 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:03:54.596 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:03:54.597 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-18T01:03:54.597 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.597 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.597 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.597 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.625 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:03:54.629 INFO:teuthology.run_tasks:Running task install... 2024-09-18T01:03:54.637 DEBUG:teuthology.task.install:project ceph 2024-09-18T01:03:54.637 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-18T01:03:54.637 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-18T01:03:54.637 INFO:teuthology.task.install:Using flavor: default 2024-09-18T01:03:54.643 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T01:03:54.643 INFO:teuthology.task.install:extra packages: [] 2024-09-18T01:03:54.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-18T01:03:54.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:03:54.646 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-18T01:03:54.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:03:54.788 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-18T01:03:54.789 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-18T01:03:54.796 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-18T01:03:54.796 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-18T01:03:54.925 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T01:03:54.925 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:03:54.925 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T01:03:54.926 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T01:03:54.926 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:03:54.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T01:03:54.952 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T01:03:54.952 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T01:03:54.953 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T01:03:54.953 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T01:03:55.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T01:03:55.020 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T01:03:55.095 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T01:03:55.096 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T01:03:55.159 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-18T01:03:55.160 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-18T01:03:55.163 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-09-18T01:03:55.164 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-09-18T01:03:55.392 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-18T01:03:55.402 INFO:teuthology.orchestra.run.smithi162.stdout:45 files removed 2024-09-18T01:03:55.419 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw 2024-09-18T01:03:55.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw 2024-09-18T01:03:55.976 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-18T01:03:55.993 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-18T01:03:56.536 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-18T01:03:56.537 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-18T01:03:59.028 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.3 MB 00:02 2024-09-18T01:03:59.960 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.3 MB 00:03 2024-09-18T01:04:01.079 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-18T01:04:04.212 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 20 MB 00:03 2024-09-18T01:04:07.404 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-18T01:04:08.463 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-18T01:04:09.429 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-18T01:04:09.988 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-18T01:04:12.567 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 23 MB 00:03 2024-09-18T01:04:12.693 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 23 MB 00:02 2024-09-18T01:04:18.047 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-18T01:04:18.124 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-18T01:04:20.256 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:04:20.276 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-09-18T01:04:20.277 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-18T01:04:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T01:04:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T01:04:20.280 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-18T01:04:20.281 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-18T01:04:20.281 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-18T01:04:20.281 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T01:04:20.281 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-18T01:04:20.281 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-09-18T01:04:20.282 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout:Install 30 Packages 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 2 Packages 2024-09-18T01:04:20.283 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:20.284 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 60 M 2024-09-18T01:04:20.285 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:04:20.333 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-18T01:04:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T01:04:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-18T01:04:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:Install 30 Packages 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 2 Packages 2024-09-18T01:04:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:20.340 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 60 M 2024-09-18T01:04:20.340 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:04:20.966 INFO:teuthology.orchestra.run.smithi162.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-18T01:04:21.024 INFO:teuthology.orchestra.run.smithi162.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 433 kB/s | 25 kB 00:00 2024-09-18T01:04:21.116 INFO:teuthology.orchestra.run.smithi162.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.0 MB/s | 751 kB 00:00 2024-09-18T01:04:21.233 INFO:teuthology.orchestra.run.smithi162.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 4.2 MB/s | 506 kB 00:00 2024-09-18T01:04:21.475 INFO:teuthology.orchestra.run.smithi162.stdout:(5/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 11 MB/s | 11 MB 00:00 2024-09-18T01:04:21.550 INFO:teuthology.orchestra.run.smithi162.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 604 kB/s | 45 kB 00:00 2024-09-18T01:04:21.625 INFO:teuthology.orchestra.run.smithi162.stdout:(7/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 14 MB/s | 5.3 MB 00:00 2024-09-18T01:04:21.650 INFO:teuthology.orchestra.run.smithi162.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 1.3 MB/s | 136 kB 00:00 2024-09-18T01:04:21.676 INFO:teuthology.orchestra.run.smithi162.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 3.1 MB/s | 162 kB 00:00 2024-09-18T01:04:21.709 INFO:teuthology.orchestra.run.smithi162.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 5.3 MB/s | 320 kB 00:00 2024-09-18T01:04:21.743 INFO:teuthology.orchestra.run.smithi162.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 4.4 MB/s | 300 kB 00:00 2024-09-18T01:04:21.768 INFO:teuthology.orchestra.run.smithi162.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-18T01:04:21.917 INFO:teuthology.orchestra.run.smithi120.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 4.0 MB/s | 5.4 MB 00:01 2024-09-18T01:04:21.967 INFO:teuthology.orchestra.run.smithi120.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 505 kB/s | 25 kB 00:00 2024-09-18T01:04:21.977 INFO:teuthology.orchestra.run.smithi162.stdout:(13/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 14 MB/s | 21 MB 00:01 2024-09-18T01:04:22.069 INFO:teuthology.orchestra.run.smithi162.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 328 kB/s | 107 kB 00:00 2024-09-18T01:04:22.084 INFO:teuthology.orchestra.run.smithi120.stdout:(3/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 7.0 MB/s | 11 MB 00:01 2024-09-18T01:04:22.094 INFO:teuthology.orchestra.run.smithi162.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-18T01:04:22.119 INFO:teuthology.orchestra.run.smithi162.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 456 kB/s | 160 kB 00:00 2024-09-18T01:04:22.126 INFO:teuthology.orchestra.run.smithi120.stdout:(4/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 4.6 MB/s | 751 kB 00:00 2024-09-18T01:04:22.151 INFO:teuthology.orchestra.run.smithi120.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 7.4 MB/s | 506 kB 00:00 2024-09-18T01:04:22.170 INFO:teuthology.orchestra.run.smithi162.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-18T01:04:22.201 INFO:teuthology.orchestra.run.smithi120.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 906 kB/s | 45 kB 00:00 2024-09-18T01:04:22.220 INFO:teuthology.orchestra.run.smithi162.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-18T01:04:22.245 INFO:teuthology.orchestra.run.smithi162.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-18T01:04:22.260 INFO:teuthology.orchestra.run.smithi120.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.3 MB/s | 136 kB 00:00 2024-09-18T01:04:22.270 INFO:teuthology.orchestra.run.smithi162.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-18T01:04:22.298 INFO:teuthology.orchestra.run.smithi162.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 462 kB/s | 36 kB 00:00 2024-09-18T01:04:22.323 INFO:teuthology.orchestra.run.smithi162.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 609 kB/s | 47 kB 00:00 2024-09-18T01:04:22.327 INFO:teuthology.orchestra.run.smithi120.stdout:(8/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 2.4 MB/s | 162 kB 00:00 2024-09-18T01:04:22.382 INFO:teuthology.orchestra.run.smithi162.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.7 MB/s | 308 kB 00:00 2024-09-18T01:04:22.402 INFO:teuthology.orchestra.run.smithi120.stdout:(9/32): python3-rados-19.1.1-330.gc2ddf972.el9. 4.2 MB/s | 320 kB 00:00 2024-09-18T01:04:22.407 INFO:teuthology.orchestra.run.smithi162.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 378 kB/s | 26 kB 00:00 2024-09-18T01:04:22.432 INFO:teuthology.orchestra.run.smithi162.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-18T01:04:22.458 INFO:teuthology.orchestra.run.smithi162.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T01:04:22.486 INFO:teuthology.orchestra.run.smithi120.stdout:(10/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 3.5 MB/s | 300 kB 00:00 2024-09-18T01:04:22.525 INFO:teuthology.orchestra.run.smithi162.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-18T01:04:22.586 INFO:teuthology.orchestra.run.smithi120.stdout:(11/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 12 MB/s | 5.3 MB 00:00 2024-09-18T01:04:22.617 INFO:teuthology.orchestra.run.smithi162.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-18T01:04:22.620 INFO:teuthology.orchestra.run.smithi120.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 743 kB/s | 99 kB 00:00 2024-09-18T01:04:22.796 INFO:teuthology.orchestra.run.smithi120.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 512 kB/s | 107 kB 00:00 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi162.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 18 MB/s | 3.4 MB 00:00 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi120.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi162.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-18T01:04:22.951 INFO:teuthology.orchestra.run.smithi162.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 22 MB/s | 3.2 MB 00:00 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi120.stdout:(15/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 8.6 MB/s | 21 MB 00:02 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi162.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi162.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi120.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 175 kB/s | 45 kB 00:00 2024-09-18T01:04:23.130 INFO:teuthology.orchestra.run.smithi120.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 565 kB/s | 42 kB 00:00 2024-09-18T01:04:23.155 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:04:23.163 INFO:teuthology.orchestra.run.smithi120.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-18T01:04:23.191 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:04:23.191 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:04:23.214 INFO:teuthology.orchestra.run.smithi120.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-09-18T01:04:23.239 INFO:teuthology.orchestra.run.smithi120.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-18T01:04:23.273 INFO:teuthology.orchestra.run.smithi120.stdout:(21/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-09-18T01:04:23.298 INFO:teuthology.orchestra.run.smithi120.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-18T01:04:23.340 INFO:teuthology.orchestra.run.smithi120.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-18T01:04:23.374 INFO:teuthology.orchestra.run.smithi120.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 380 kB/s | 26 kB 00:00 2024-09-18T01:04:23.399 INFO:teuthology.orchestra.run.smithi120.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 856 kB/s | 49 kB 00:00 2024-09-18T01:04:23.424 INFO:teuthology.orchestra.run.smithi120.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T01:04:23.475 INFO:teuthology.orchestra.run.smithi120.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-18T01:04:23.533 INFO:teuthology.orchestra.run.smithi120.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-18T01:04:23.800 INFO:teuthology.orchestra.run.smithi120.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 13 MB/s | 3.4 MB 00:00 2024-09-18T01:04:23.842 INFO:teuthology.orchestra.run.smithi120.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-09-18T01:04:23.851 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:04:23.852 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:04:23.993 INFO:teuthology.orchestra.run.smithi120.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 16 MB/s | 3.2 MB 00:00 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi120.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi120.stdout:Total 16 MB/s | 60 MB 00:03 2024-09-18T01:04:24.179 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:04:24.214 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:04:24.214 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:04:24.894 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:04:24.895 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:04:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:04:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-18T01:04:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-18T01:04:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-18T01:04:25.562 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T01:04:25.588 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T01:04:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T01:04:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T01:04:26.181 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-18T01:04:26.267 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:26.361 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:04:26.459 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-18T01:04:26.475 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-18T01:04:26.494 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T01:04:26.507 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-18T01:04:26.532 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-18T01:04:26.555 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T01:04:26.557 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T01:04:26.582 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T01:04:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-18T01:04:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-18T01:04:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:04:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T01:04:26.785 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-18T01:04:26.835 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T01:04:26.939 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-18T01:04:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-18T01:04:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-18T01:04:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-18T01:04:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-18T01:04:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-18T01:04:27.226 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-18T01:04:27.292 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:27.425 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-18T01:04:27.470 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T01:04:27.488 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-18T01:04:27.519 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T01:04:27.569 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T01:04:27.590 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T01:04:27.599 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-18T01:04:27.635 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-18T01:04:27.635 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-18T01:04:27.680 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-18T01:04:27.749 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:04:27.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:27.827 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:27.862 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-18T01:04:28.004 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-18T01:04:28.052 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-18T01:04:28.086 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-18T01:04:28.115 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-18T01:04:28.116 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-18T01:04:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T01:04:28.157 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-18T01:04:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T01:04:28.288 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-18T01:04:28.317 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-18T01:04:28.520 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T01:04:28.609 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T01:04:28.649 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-18T01:04:28.685 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-18T01:04:29.173 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-18T01:04:29.356 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T01:04:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T01:04:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-18T01:04:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-18T01:04:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:31.187 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:31.225 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T01:04:31.319 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T01:04:31.319 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T01:04:31.319 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T01:04:31.319 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:31.353 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T01:04:32.709 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T01:04:32.786 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T01:04:32.787 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T01:04:32.787 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T01:04:32.787 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:32.852 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:41.373 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:41.373 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-09-18T01:04:41.373 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-09-18T01:04:41.373 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-09-18T01:04:41.373 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-09-18T01:04:41.374 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-09-18T01:04:41.374 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-09-18T01:04:41.374 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-09-18T01:04:41.374 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:42.376 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:04:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:04:42.401 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:04:42.401 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T01:04:42.401 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:04:42.402 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:04:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:04:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:04:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-18T01:04:43.039 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-18T01:04:43.040 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-18T01:04:43.040 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-18T01:04:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-18T01:04:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-18T01:04:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-18T01:04:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-18T01:04:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-18T01:04:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-18T01:04:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-18T01:04:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-18T01:04:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-18T01:04:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-18T01:04:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:04:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:04:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:04:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:44.833 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:44.833 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:44.833 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-09-18T01:04:44.833 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T01:04:44.834 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.835 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:44.836 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:04:45.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-test 2024-09-18T01:04:45.388 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:04:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-18T01:04:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-18T01:04:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:27 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:04:45.919 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-18T01:04:45.920 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-18T01:04:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:45.922 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 49 M 2024-09-18T01:04:45.922 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 209 M 2024-09-18T01:04:45.922 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-18T01:04:46.293 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 664 kB/s | 163 kB 00:00 2024-09-18T01:04:46.318 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-18T01:04:46.514 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.515 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T01:04:46.516 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T01:04:46.517 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:46.518 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:04:46.904 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-test 2024-09-18T01:04:47.478 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:29 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:04:47.593 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-18T01:04:47.595 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:04:47.596 INFO:teuthology.orchestra.run.smithi162.stdout:Total 29 MB/s | 49 MB 00:01 2024-09-18T01:04:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:04:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:47.612 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:04:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-09-18T01:04:47.613 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 49 M 2024-09-18T01:04:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 209 M 2024-09-18T01:04:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:04:47.623 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:04:47.643 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:04:47.643 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:04:47.837 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-18T01:04:47.879 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-18T01:04:47.929 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 754 kB/s | 163 kB 00:00 2024-09-18T01:04:48.033 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:04:48.033 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:04:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T01:04:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T01:04:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:04:49.271 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-18T01:04:49.273 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:04:49.274 INFO:teuthology.orchestra.run.smithi120.stdout:Total 30 MB/s | 49 MB 00:01 2024-09-18T01:04:49.301 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:04:49.322 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:04:49.322 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:04:49.705 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:04:49.706 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:04:50.234 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:04:50.325 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T01:04:50.371 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T01:04:50.399 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:04:53.369 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:04:53.454 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:04:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:04:55.106 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:04:55.930 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:04:55.930 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-18T01:04:55.930 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-18T01:04:55.930 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T01:04:56.224 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:56.225 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:04:56.511 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph 2024-09-18T01:04:57.102 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:39 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:04:57.237 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-18T01:04:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T01:04:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T01:04:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:04:57.243 INFO:teuthology.orchestra.run.smithi162.stdout:Install 37 Packages 2024-09-18T01:04:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:04:57.244 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 30 M 2024-09-18T01:04:57.244 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 108 M 2024-09-18T01:04:57.244 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:04:57.585 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:04:57.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-18T01:04:57.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-18T01:04:57.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T01:04:57.639 INFO:teuthology.orchestra.run.smithi162.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 55 kB/s | 6.5 kB 00:00 2024-09-18T01:04:57.855 INFO:teuthology.orchestra.run.smithi162.stdout:(2/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 7.1 MB/s | 2.4 MB 00:00 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:57.863 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:04:57.897 INFO:teuthology.orchestra.run.smithi162.stdout:(3/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.0 MB/s | 1.5 MB 00:00 2024-09-18T01:04:58.010 INFO:teuthology.orchestra.run.smithi162.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.2 MB/s | 249 kB 00:00 2024-09-18T01:04:58.077 INFO:teuthology.orchestra.run.smithi162.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-18T01:04:58.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph 2024-09-18T01:04:58.402 INFO:teuthology.orchestra.run.smithi162.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-18T01:04:58.727 INFO:teuthology.orchestra.run.smithi162.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 353 kB/s | 253 kB 00:00 2024-09-18T01:04:58.760 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:40 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:04:58.844 INFO:teuthology.orchestra.run.smithi162.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-18T01:04:58.878 INFO:teuthology.orchestra.run.smithi162.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 224 kB/s | 106 kB 00:00 2024-09-18T01:04:58.900 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:04:58.902 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:58.902 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:04:58.902 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:58.902 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T01:04:58.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T01:04:58.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:04:58.905 INFO:teuthology.orchestra.run.smithi120.stdout:Install 37 Packages 2024-09-18T01:04:58.906 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:04:58.906 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 30 M 2024-09-18T01:04:58.906 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 108 M 2024-09-18T01:04:58.906 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:04:58.953 INFO:teuthology.orchestra.run.smithi162.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-09-18T01:04:59.004 INFO:teuthology.orchestra.run.smithi162.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-09-18T01:04:59.079 INFO:teuthology.orchestra.run.smithi162.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-18T01:04:59.433 INFO:teuthology.orchestra.run.smithi120.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 56 kB/s | 6.5 kB 00:00 2024-09-18T01:04:59.626 INFO:teuthology.orchestra.run.smithi120.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-18T01:04:59.646 INFO:teuthology.orchestra.run.smithi162.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 61 kB/s | 35 kB 00:00 2024-09-18T01:04:59.679 INFO:teuthology.orchestra.run.smithi162.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-09-18T01:04:59.684 INFO:teuthology.orchestra.run.smithi120.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-18T01:04:59.747 INFO:teuthology.orchestra.run.smithi162.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-18T01:04:59.768 INFO:teuthology.orchestra.run.smithi120.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.9 MB/s | 249 kB 00:00 2024-09-18T01:04:59.860 INFO:teuthology.orchestra.run.smithi120.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-18T01:05:00.185 INFO:teuthology.orchestra.run.smithi120.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-18T01:05:00.313 INFO:teuthology.orchestra.run.smithi162.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 116 kB/s | 77 kB 00:00 2024-09-18T01:05:00.339 INFO:teuthology.orchestra.run.smithi162.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-09-18T01:05:00.364 INFO:teuthology.orchestra.run.smithi162.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-18T01:05:00.431 INFO:teuthology.orchestra.run.smithi162.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 654 kB/s | 43 kB 00:00 2024-09-18T01:05:00.456 INFO:teuthology.orchestra.run.smithi162.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-18T01:05:00.482 INFO:teuthology.orchestra.run.smithi162.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 135 kB/s | 22 kB 00:00 2024-09-18T01:05:00.515 INFO:teuthology.orchestra.run.smithi162.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-18T01:05:00.540 INFO:teuthology.orchestra.run.smithi162.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2024-09-18T01:05:00.566 INFO:teuthology.orchestra.run.smithi162.stdout:(24/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 127 kB/s | 11 kB 00:00 2024-09-18T01:05:00.591 INFO:teuthology.orchestra.run.smithi162.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 239 kB/s | 18 kB 00:00 2024-09-18T01:05:00.616 INFO:teuthology.orchestra.run.smithi162.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-18T01:05:00.641 INFO:teuthology.orchestra.run.smithi162.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-18T01:05:00.668 INFO:teuthology.orchestra.run.smithi162.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-18T01:05:00.692 INFO:teuthology.orchestra.run.smithi162.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-18T01:05:00.717 INFO:teuthology.orchestra.run.smithi162.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-18T01:05:00.743 INFO:teuthology.orchestra.run.smithi162.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-18T01:05:00.768 INFO:teuthology.orchestra.run.smithi162.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-18T01:05:00.793 INFO:teuthology.orchestra.run.smithi162.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-18T01:05:00.821 INFO:teuthology.orchestra.run.smithi162.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-18T01:05:00.835 INFO:teuthology.orchestra.run.smithi120.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 237 kB/s | 253 kB 00:01 2024-09-18T01:05:00.852 INFO:teuthology.orchestra.run.smithi162.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-18T01:05:00.869 INFO:teuthology.orchestra.run.smithi120.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 159 kB/s | 106 kB 00:00 2024-09-18T01:05:00.869 INFO:teuthology.orchestra.run.smithi162.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-18T01:05:00.902 INFO:teuthology.orchestra.run.smithi162.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-18T01:05:00.907 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:00.907 INFO:teuthology.orchestra.run.smithi162.stdout:Total 8.3 MB/s | 30 MB 00:03 2024-09-18T01:05:00.936 INFO:teuthology.orchestra.run.smithi120.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-18T01:05:01.059 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:01.099 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:01.099 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:01.570 INFO:teuthology.orchestra.run.smithi120.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 184 kB/s | 135 kB 00:00 2024-09-18T01:05:01.578 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:01.579 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:01.620 INFO:teuthology.orchestra.run.smithi120.stdout:(11/37): python3-requests-2.25.1-8.el9.noarch.r 166 kB/s | 125 kB 00:00 2024-09-18T01:05:01.670 INFO:teuthology.orchestra.run.smithi120.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 293 kB/s | 215 kB 00:00 2024-09-18T01:05:01.787 INFO:teuthology.orchestra.run.smithi120.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 299 kB/s | 35 kB 00:00 2024-09-18T01:05:01.812 INFO:teuthology.orchestra.run.smithi120.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 778 kB/s | 188 kB 00:00 2024-09-18T01:05:01.854 INFO:teuthology.orchestra.run.smithi120.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 736 kB/s | 172 kB 00:00 2024-09-18T01:05:01.880 INFO:teuthology.orchestra.run.smithi120.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-09-18T01:05:01.913 INFO:teuthology.orchestra.run.smithi120.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-18T01:05:01.939 INFO:teuthology.orchestra.run.smithi120.stdout:(18/37): python3-packaging-20.9-5.el9.noarch.rp 510 kB/s | 77 kB 00:00 2024-09-18T01:05:01.981 INFO:teuthology.orchestra.run.smithi120.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-18T01:05:02.023 INFO:teuthology.orchestra.run.smithi120.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 521 kB/s | 43 kB 00:00 2024-09-18T01:05:02.056 INFO:teuthology.orchestra.run.smithi120.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-09-18T01:05:02.082 INFO:teuthology.orchestra.run.smithi120.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 425 kB/s | 11 kB 00:00 2024-09-18T01:05:02.107 INFO:teuthology.orchestra.run.smithi120.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 710 kB/s | 18 kB 00:00 2024-09-18T01:05:02.132 INFO:teuthology.orchestra.run.smithi120.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-18T01:05:02.158 INFO:teuthology.orchestra.run.smithi120.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-18T01:05:02.183 INFO:teuthology.orchestra.run.smithi120.stdout:(26/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-18T01:05:02.208 INFO:teuthology.orchestra.run.smithi120.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 256 kB/s | 19 kB 00:00 2024-09-18T01:05:02.233 INFO:teuthology.orchestra.run.smithi120.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-18T01:05:02.259 INFO:teuthology.orchestra.run.smithi120.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-09-18T01:05:02.284 INFO:teuthology.orchestra.run.smithi120.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-18T01:05:02.309 INFO:teuthology.orchestra.run.smithi120.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-18T01:05:02.343 INFO:teuthology.orchestra.run.smithi120.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-18T01:05:02.364 INFO:teuthology.orchestra.run.smithi120.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 663 kB/s | 36 kB 00:00 2024-09-18T01:05:02.389 INFO:teuthology.orchestra.run.smithi120.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-18T01:05:02.423 INFO:teuthology.orchestra.run.smithi120.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 599 kB/s | 20 kB 00:00 2024-09-18T01:05:02.448 INFO:teuthology.orchestra.run.smithi120.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-18T01:05:02.473 INFO:teuthology.orchestra.run.smithi120.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-18T01:05:02.478 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:02.478 INFO:teuthology.orchestra.run.smithi120.stdout:Total 8.5 MB/s | 30 MB 00:03 2024-09-18T01:05:02.631 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:02.671 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:02.672 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-18T01:05:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-18T01:05:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-18T01:05:03.165 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:03.166 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-18T01:05:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-18T01:05:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-18T01:05:03.390 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-18T01:05:03.469 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-18T01:05:03.510 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-18T01:05:03.545 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-18T01:05:03.583 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-18T01:05:03.626 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-18T01:05:03.660 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-18T01:05:03.700 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-18T01:05:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-18T01:05:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-18T01:05:03.832 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-18T01:05:03.864 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-18T01:05:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-18T01:05:04.093 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-18T01:05:04.181 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-18T01:05:04.235 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-18T01:05:04.272 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-18T01:05:04.321 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-18T01:05:04.385 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-18T01:05:04.543 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:04.584 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-18T01:05:04.609 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-18T01:05:04.636 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-18T01:05:04.643 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-18T01:05:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-18T01:05:04.714 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-18T01:05:04.741 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-18T01:05:04.786 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-18T01:05:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-18T01:05:04.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-18T01:05:04.852 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-18T01:05:04.912 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-18T01:05:04.975 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-18T01:05:04.998 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-18T01:05:05.039 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-18T01:05:05.074 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-18T01:05:05.104 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-18T01:05:05.138 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-18T01:05:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-18T01:05:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:05:05.172 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-18T01:05:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:05:05.197 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:05.197 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T01:05:05.197 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:05:05.197 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:05:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:05.212 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-18T01:05:05.247 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-18T01:05:05.290 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-18T01:05:05.339 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-18T01:05:05.379 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-18T01:05:05.453 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-18T01:05:05.592 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-18T01:05:05.670 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-18T01:05:05.725 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-18T01:05:05.763 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-18T01:05:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-18T01:05:05.873 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-18T01:05:06.062 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-18T01:05:06.115 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-18T01:05:06.192 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-18T01:05:06.285 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-18T01:05:06.341 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-18T01:05:06.458 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-18T01:05:06.622 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-18T01:05:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:05:06.652 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:05:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:05:06.666 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:06.666 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T01:05:06.666 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:05:06.666 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:05:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:06.678 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:05:06.679 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:06.679 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T01:05:06.679 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:05:06.679 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:05:06.679 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:07.098 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:05:07.121 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:05:07.121 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:07.121 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T01:05:07.121 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:05:07.121 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:05:07.122 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:05:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:05:07.380 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:07.380 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T01:05:07.381 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:05:07.381 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:05:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:07.445 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:05:08.111 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:05:08.138 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:05:08.138 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:08.138 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T01:05:08.139 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:05:08.139 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:05:08.139 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:08.579 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:05:08.603 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:08.830 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:05:08.853 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:05:08.854 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:08.854 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T01:05:08.854 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:05:08.854 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:05:08.854 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:08.919 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-18T01:05:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-18T01:05:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-18T01:05:09.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-18T01:05:09.198 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T01:05:10.285 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T01:05:10.286 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T01:05:10.287 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:10.288 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:05:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-18T01:05:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-18T01:05:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-18T01:05:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-18T01:05:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-18T01:05:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-18T01:05:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-18T01:05:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-18T01:05:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-18T01:05:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-18T01:05:10.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-base 2024-09-18T01:05:11.270 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:53 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:11.359 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:11.401 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:11.402 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:11.402 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-18T01:05:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T01:05:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T01:05:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T01:05:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T01:05:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T01:05:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:11.479 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:11.480 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install cephadm 2024-09-18T01:05:11.827 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-base 2024-09-18T01:05:12.060 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:12.192 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:12.193 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:12.193 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:12.193 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 765 k 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 771 k 2024-09-18T01:05:12.194 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:12.406 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:12.473 INFO:teuthology.orchestra.run.smithi162.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-18T01:05:12.473 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:12.474 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-18T01:05:12.474 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:12.479 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:12.479 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:12.486 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:12.486 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:12.494 INFO:teuthology.orchestra.run.smithi120.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:12.537 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:12.537 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:05:12.537 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:12.556 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:12.603 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install cephadm 2024-09-18T01:05:12.991 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:13.086 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:13.229 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:13.313 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:13.314 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:13.314 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:13.314 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:13.314 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:13.314 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 765 k 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 771 k 2024-09-18T01:05:13.315 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:13.466 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:13.590 INFO:teuthology.orchestra.run.smithi120.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-18T01:05:13.590 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:13.590 INFO:teuthology.orchestra.run.smithi120.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-18T01:05:13.591 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:13.596 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:13.596 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:13.602 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:13.602 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:13.674 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:13.706 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:13.706 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:13.706 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:13.706 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:13.706 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:13.707 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:13.980 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T01:05:14.067 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:14.144 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:14.519 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:14.574 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:14.710 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:14.711 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 145 k 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 435 k 2024-09-18T01:05:14.712 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:14.755 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:14.755 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:14.756 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:14.756 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:14.756 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:14.756 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:14.921 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 698 kB/s | 145 kB 00:00 2024-09-18T01:05:14.922 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:14.922 INFO:teuthology.orchestra.run.smithi162.stdout:Total 691 kB/s | 145 kB 00:00 2024-09-18T01:05:14.922 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:14.932 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:14.932 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:14.975 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T01:05:14.987 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:14.987 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:15.214 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:15.239 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:15.240 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:15.240 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T01:05:15.240 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:15.571 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:57 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:15.705 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repo Size 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:15.706 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 145 k 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 435 k 2024-09-18T01:05:15.707 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:15.922 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 677 kB/s | 145 kB 00:00 2024-09-18T01:05:15.922 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:15.923 INFO:teuthology.orchestra.run.smithi120.stdout:Total 671 kB/s | 145 kB 00:00 2024-09-18T01:05:15.923 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:15.930 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:15.930 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:15.988 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:15.988 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:16.140 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:16.180 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:16.181 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:16.248 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:16.248 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:16.248 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T01:05:16.248 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:16.443 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr 2024-09-18T01:05:17.037 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:59 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:17.130 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:17.172 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:17.173 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:17.173 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:17.179 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:17.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T01:05:17.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-mgr 2024-09-18T01:05:17.816 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:59 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:17.947 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:17.948 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:17.948 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:17.948 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T01:05:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout:Install 12 Packages 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:17.950 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 8.7 M 2024-09-18T01:05:17.951 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 104 M 2024-09-18T01:05:17.951 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:18.003 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:00 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:18.095 INFO:teuthology.orchestra.run.smithi120.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:18.137 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:18.138 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:05:18.138 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:18.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T01:05:18.678 INFO:teuthology.orchestra.run.smithi162.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 108 kB/s | 32 kB 00:00 2024-09-18T01:05:18.702 INFO:teuthology.orchestra.run.smithi162.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 51 kB/s | 16 kB 00:00 2024-09-18T01:05:18.779 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:00 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:18.819 INFO:teuthology.orchestra.run.smithi162.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-18T01:05:18.870 INFO:teuthology.orchestra.run.smithi162.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-18T01:05:18.911 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:18.912 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T01:05:18.913 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout:Install 12 Packages 2024-09-18T01:05:18.914 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:18.915 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 8.7 M 2024-09-18T01:05:18.915 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 104 M 2024-09-18T01:05:18.915 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:19.012 INFO:teuthology.orchestra.run.smithi162.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.8 MB/s | 548 kB 00:00 2024-09-18T01:05:19.045 INFO:teuthology.orchestra.run.smithi162.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.7 MB/s | 863 kB 00:00 2024-09-18T01:05:19.071 INFO:teuthology.orchestra.run.smithi162.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-18T01:05:19.138 INFO:teuthology.orchestra.run.smithi162.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-09-18T01:05:19.172 INFO:teuthology.orchestra.run.smithi162.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 927 kB/s | 31 kB 00:00 2024-09-18T01:05:19.205 INFO:teuthology.orchestra.run.smithi162.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-18T01:05:19.406 INFO:teuthology.orchestra.run.smithi162.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.7 MB/s | 2.0 MB 00:00 2024-09-18T01:05:19.860 INFO:teuthology.orchestra.run.smithi120.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 158 kB/s | 32 kB 00:00 2024-09-18T01:05:19.885 INFO:teuthology.orchestra.run.smithi120.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 73 kB/s | 16 kB 00:00 2024-09-18T01:05:20.672 INFO:teuthology.orchestra.run.smithi162.stdout:(12/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf97 1.6 MB/s | 3.6 MB 00:02 2024-09-18T01:05:20.677 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:20.677 INFO:teuthology.orchestra.run.smithi162.stdout:Total 3.2 MB/s | 8.7 MB 00:02 2024-09-18T01:05:20.727 INFO:teuthology.orchestra.run.smithi120.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 320 kB/s | 269 kB 00:00 2024-09-18T01:05:20.729 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:20.748 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:20.748 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:20.802 INFO:teuthology.orchestra.run.smithi120.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 3.1 MB/s | 3.6 MB 00:01 2024-09-18T01:05:20.836 INFO:teuthology.orchestra.run.smithi120.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-09-18T01:05:20.894 INFO:teuthology.orchestra.run.smithi120.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 385 kB/s | 19 kB 00:00 2024-09-18T01:05:20.954 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:20.954 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:20.978 INFO:teuthology.orchestra.run.smithi120.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.0 MB/s | 548 kB 00:00 2024-09-18T01:05:21.028 INFO:teuthology.orchestra.run.smithi120.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-18T01:05:21.062 INFO:teuthology.orchestra.run.smithi120.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 925 kB/s | 31 kB 00:00 2024-09-18T01:05:21.129 INFO:teuthology.orchestra.run.smithi120.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-18T01:05:21.421 INFO:teuthology.orchestra.run.smithi120.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.9 MB/s | 2.0 MB 00:00 2024-09-18T01:05:21.486 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:21.564 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-18T01:05:21.600 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-18T01:05:21.646 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-18T01:05:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-18T01:05:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-18T01:05:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-18T01:05:22.005 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-18T01:05:22.058 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-18T01:05:22.086 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:05:22.117 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-18T01:05:22.329 INFO:teuthology.orchestra.run.smithi120.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 539 kB/s | 863 kB 00:01 2024-09-18T01:05:22.335 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:22.335 INFO:teuthology.orchestra.run.smithi120.stdout:Total 2.5 MB/s | 8.7 MB 00:03 2024-09-18T01:05:22.386 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:22.405 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:22.405 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:22.627 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:22.627 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:23.164 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:23.170 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-18T01:05:23.210 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:05:23.246 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-18T01:05:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-18T01:05:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-18T01:05:23.376 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-18T01:05:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-18T01:05:23.601 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-18T01:05:23.695 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-18T01:05:23.748 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-18T01:05:23.777 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:05:23.808 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-18T01:05:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:05:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-18T01:05:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-18T01:05:24.865 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-18T01:05:24.909 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:24.986 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:25.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T01:05:25.829 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:07 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:25.960 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-18T01:05:25.961 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T01:05:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:Install 12 Packages 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 38 M 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 212 M 2024-09-18T01:05:25.963 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-18T01:05:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-18T01:05:26.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-18T01:05:26.595 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:26.595 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:26.595 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T01:05:26.595 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:26.596 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:26.640 INFO:teuthology.orchestra.run.smithi162.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-18T01:05:26.690 INFO:teuthology.orchestra.run.smithi162.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-18T01:05:26.790 INFO:teuthology.orchestra.run.smithi162.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 33 kB 00:00 2024-09-18T01:05:26.840 INFO:teuthology.orchestra.run.smithi162.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-18T01:05:26.858 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T01:05:26.907 INFO:teuthology.orchestra.run.smithi162.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 578 kB/s | 38 kB 00:00 2024-09-18T01:05:26.999 INFO:teuthology.orchestra.run.smithi162.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-33 15 MB/s | 7.4 MB 00:00 2024-09-18T01:05:27.058 INFO:teuthology.orchestra.run.smithi162.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.1 MB/s | 3.0 MB 00:00 2024-09-18T01:05:27.192 INFO:teuthology.orchestra.run.smithi162.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-18T01:05:27.284 INFO:teuthology.orchestra.run.smithi162.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-18T01:05:27.308 INFO:teuthology.orchestra.run.smithi162.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-18T01:05:27.342 INFO:teuthology.orchestra.run.smithi162.stdout:(11/12): python3-devel-3.9.19-8.el9.x86_64.rpm 714 kB/s | 245 kB 00:00 2024-09-18T01:05:27.440 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:09 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:27.571 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:27.572 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-18T01:05:27.572 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:27.572 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-18T01:05:27.572 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T01:05:27.573 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout:Install 12 Packages 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:27.574 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 38 M 2024-09-18T01:05:27.575 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 212 M 2024-09-18T01:05:27.575 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:27.609 INFO:teuthology.orchestra.run.smithi162.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-09-18T01:05:27.610 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:27.610 INFO:teuthology.orchestra.run.smithi162.stdout:Total 23 MB/s | 38 MB 00:01 2024-09-18T01:05:27.960 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:27.987 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:27.987 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:28.271 INFO:teuthology.orchestra.run.smithi120.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-18T01:05:28.381 INFO:teuthology.orchestra.run.smithi120.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-09-18T01:05:28.422 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:28.422 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:28.640 INFO:teuthology.orchestra.run.smithi120.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 227 kB/s | 191 kB 00:00 2024-09-18T01:05:28.690 INFO:teuthology.orchestra.run.smithi120.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.7 MB/s | 3.0 MB 00:00 2024-09-18T01:05:28.740 INFO:teuthology.orchestra.run.smithi120.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-18T01:05:28.765 INFO:teuthology.orchestra.run.smithi120.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-09-18T01:05:28.808 INFO:teuthology.orchestra.run.smithi120.stdout:(7/12): libgfortran-11.5.0-2.el9.x86_64.rpm 792 kB/s | 801 kB 00:01 2024-09-18T01:05:28.908 INFO:teuthology.orchestra.run.smithi120.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-18T01:05:28.966 INFO:teuthology.orchestra.run.smithi120.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-18T01:05:29.033 INFO:teuthology.orchestra.run.smithi120.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 17 MB/s | 4.9 MB 00:00 2024-09-18T01:05:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T01:05:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:05:29.451 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:05:29.484 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T01:05:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T01:05:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T01:05:29.717 INFO:teuthology.orchestra.run.smithi120.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-18T01:05:29.776 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T01:05:29.792 INFO:teuthology.orchestra.run.smithi120.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.2 MB/s | 5.1 MB 00:00 2024-09-18T01:05:29.793 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:29.793 INFO:teuthology.orchestra.run.smithi120.stdout:Total 17 MB/s | 38 MB 00:02 2024-09-18T01:05:30.147 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:30.175 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:30.175 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:30.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T01:05:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T01:05:30.608 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:30.608 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T01:05:31.483 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:05:31.589 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:05:31.633 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T01:05:31.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T01:05:31.819 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:05:31.849 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T01:05:31.926 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T01:05:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T01:05:32.695 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T01:05:32.850 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T01:05:32.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:05:33.946 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:05:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T01:05:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:05:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:05:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:05:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T01:05:36.019 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T01:05:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T01:05:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T01:05:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:36.021 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:36.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-rook 2024-09-18T01:05:37.028 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:19 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:37.161 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T01:05:37.162 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:37.163 INFO:teuthology.orchestra.run.smithi162.stdout:Install 13 Packages 2024-09-18T01:05:37.164 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:37.164 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.2 M 2024-09-18T01:05:37.164 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 26 M 2024-09-18T01:05:37.164 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:37.419 INFO:teuthology.orchestra.run.smithi162.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 161 kB/s | 19 kB 00:00 2024-09-18T01:05:37.436 INFO:teuthology.orchestra.run.smithi162.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 188 kB/s | 26 kB 00:00 2024-09-18T01:05:37.453 INFO:teuthology.orchestra.run.smithi162.stdout:(3/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 322 kB/s | 49 kB 00:00 2024-09-18T01:05:37.562 INFO:teuthology.orchestra.run.smithi162.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-18T01:05:37.587 INFO:teuthology.orchestra.run.smithi162.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-18T01:05:37.621 INFO:teuthology.orchestra.run.smithi162.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-09-18T01:05:37.655 INFO:teuthology.orchestra.run.smithi162.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 479 kB/s | 32 kB 00:00 2024-09-18T01:05:37.680 INFO:teuthology.orchestra.run.smithi162.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-18T01:05:37.748 INFO:teuthology.orchestra.run.smithi162.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.4 MB/s | 220 kB 00:00 2024-09-18T01:05:37.772 INFO:teuthology.orchestra.run.smithi162.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-18T01:05:37.805 INFO:teuthology.orchestra.run.smithi162.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 8.6 MB/s | 1.0 MB 00:00 2024-09-18T01:05:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:05:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T01:05:37.831 INFO:teuthology.orchestra.run.smithi162.stdout:(12/13): python3-pyasn1-modules-0.4.8-6.el9.noa 739 kB/s | 279 kB 00:00 2024-09-18T01:05:37.865 INFO:teuthology.orchestra.run.smithi162.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 972 kB/s | 90 kB 00:00 2024-09-18T01:05:37.866 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:37.866 INFO:teuthology.orchestra.run.smithi162.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-18T01:05:37.931 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:37.946 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:37.947 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:38.036 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:38.037 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:38.236 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T01:05:38.236 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T01:05:38.237 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T01:05:38.238 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T01:05:38.238 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T01:05:38.238 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:38.238 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:38.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-mgr-rook 2024-09-18T01:05:38.711 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T01:05:38.757 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T01:05:38.821 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T01:05:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T01:05:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T01:05:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T01:05:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T01:05:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T01:05:39.199 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:21 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:39.331 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T01:05:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:39.333 INFO:teuthology.orchestra.run.smithi120.stdout:Install 13 Packages 2024-09-18T01:05:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:39.334 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 2.2 M 2024-09-18T01:05:39.334 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 26 M 2024-09-18T01:05:39.334 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:39.829 INFO:teuthology.orchestra.run.smithi120.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 313 kB/s | 49 kB 00:00 2024-09-18T01:05:39.888 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T01:05:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T01:05:39.946 INFO:teuthology.orchestra.run.smithi120.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 93 kB/s | 26 kB 00:00 2024-09-18T01:05:39.964 INFO:teuthology.orchestra.run.smithi120.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 66 kB/s | 19 kB 00:00 2024-09-18T01:05:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T01:05:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T01:05:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:05:40.314 INFO:teuthology.orchestra.run.smithi120.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 432 kB/s | 159 kB 00:00 2024-09-18T01:05:40.389 INFO:teuthology.orchestra.run.smithi120.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 397 kB/s | 222 kB 00:00 2024-09-18T01:05:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 576 kB/s | 279 kB 00:00 2024-09-18T01:05:40.473 INFO:teuthology.orchestra.run.smithi120.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 337 kB/s | 54 kB 00:00 2024-09-18T01:05:40.507 INFO:teuthology.orchestra.run.smithi120.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 275 kB/s | 32 kB 00:00 2024-09-18T01:05:40.540 INFO:teuthology.orchestra.run.smithi120.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-18T01:05:40.574 INFO:teuthology.orchestra.run.smithi120.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-18T01:05:40.608 INFO:teuthology.orchestra.run.smithi120.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.7 MB/s | 220 kB 00:00 2024-09-18T01:05:40.633 INFO:teuthology.orchestra.run.smithi120.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-18T01:05:40.683 INFO:teuthology.orchestra.run.smithi120.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-09-18T01:05:40.686 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:40.687 INFO:teuthology.orchestra.run.smithi120.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-09-18T01:05:40.759 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:40.864 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:40.865 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T01:05:40.896 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T01:05:40.897 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T01:05:40.897 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T01:05:40.897 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T01:05:40.897 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T01:05:40.897 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T01:05:41.232 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T01:05:41.233 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:41.234 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:41.370 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:41.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T01:05:41.548 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T01:05:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T01:05:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T01:05:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T01:05:41.752 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T01:05:41.861 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T01:05:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T01:05:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T01:05:42.089 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:24 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:42.221 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:42.221 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T01:05:42.222 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout:Install 6 Packages 2024-09-18T01:05:42.223 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:42.224 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 7.0 M 2024-09-18T01:05:42.224 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 33 M 2024-09-18T01:05:42.224 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:42.738 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T01:05:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T01:05:42.827 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T01:05:42.876 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T01:05:42.906 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:05:43.002 INFO:teuthology.orchestra.run.smithi162.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi162.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 711 kB/s | 146 kB 00:00 2024-09-18T01:05:43.194 INFO:teuthology.orchestra.run.smithi162.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-18T01:05:43.220 INFO:teuthology.orchestra.run.smithi162.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 406 kB/s | 58 kB 00:00 2024-09-18T01:05:43.262 INFO:teuthology.orchestra.run.smithi162.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.1 MB/s | 548 kB 00:00 2024-09-18T01:05:43.304 INFO:teuthology.orchestra.run.smithi162.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 686 kB/s | 74 kB 00:00 2024-09-18T01:05:43.307 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:43.307 INFO:teuthology.orchestra.run.smithi162.stdout:Total 6.5 MB/s | 7.0 MB 00:01 2024-09-18T01:05:43.377 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:43.388 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:43.388 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:43.488 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:43.489 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:43.665 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:05:43.665 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-18T01:05:43.665 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T01:05:43.665 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T01:05:43.666 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T01:05:43.796 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T01:05:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:44.018 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T01:05:44.269 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T01:05:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T01:05:44.659 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T01:05:44.730 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T01:05:44.768 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:05:44.846 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:26 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:44.979 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:44.979 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:44.980 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout:Install 6 Packages 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 7.0 M 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 33 M 2024-09-18T01:05:44.981 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:45.387 INFO:teuthology.orchestra.run.smithi120.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 711 kB/s | 146 kB 00:00 2024-09-18T01:05:45.412 INFO:teuthology.orchestra.run.smithi120.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-18T01:05:45.471 INFO:teuthology.orchestra.run.smithi120.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 983 kB/s | 58 kB 00:00 2024-09-18T01:05:45.505 INFO:teuthology.orchestra.run.smithi120.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-18T01:05:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:05:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-18T01:05:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T01:05:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T01:05:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T01:05:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T01:05:45.572 INFO:teuthology.orchestra.run.smithi120.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.9 MB/s | 548 kB 00:00 2024-09-18T01:05:45.831 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:45.832 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:46.006 INFO:teuthology.orchestra.run.smithi120.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.2 MB/s | 6.0 MB 00:00 2024-09-18T01:05:46.007 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:46.008 INFO:teuthology.orchestra.run.smithi120.stdout:Total 6.8 MB/s | 7.0 MB 00:01 2024-09-18T01:05:46.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-fuse 2024-09-18T01:05:46.081 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:46.091 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:46.091 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:46.191 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:46.191 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:46.491 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T01:05:46.657 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:28 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T01:05:46.791 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:46.792 INFO:teuthology.orchestra.run.smithi162.stdout:Install 2 Packages 2024-09-18T01:05:46.793 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:46.793 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 977 k 2024-09-18T01:05:46.793 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 2.9 M 2024-09-18T01:05:46.793 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:47.073 INFO:teuthology.orchestra.run.smithi162.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 664 kB/s | 80 kB 00:00 2024-09-18T01:05:47.224 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T01:05:47.310 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T01:05:47.375 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T01:05:47.412 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:05:47.640 INFO:teuthology.orchestra.run.smithi162.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 1.3 MB/s | 897 kB 00:00 2024-09-18T01:05:47.640 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:47.640 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.1 MB/s | 977 kB 00:00 2024-09-18T01:05:47.651 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:47.673 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:47.674 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:47.755 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:47.755 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:48.086 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T01:05:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:05:48.207 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:05:48.208 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-18T01:05:48.208 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T01:05:48.208 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T01:05:48.208 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T01:05:48.208 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T01:05:48.475 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T01:05:48.476 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T01:05:48.476 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T01:05:48.476 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:48.476 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:48.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-fuse 2024-09-18T01:05:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:05:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-18T01:05:49.139 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T01:05:49.139 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:49.139 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:49.140 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T01:05:49.140 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:49.140 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:49.319 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:49.436 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-volume 2024-09-18T01:05:49.451 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T01:05:49.452 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:Install 2 Packages 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 977 k 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 2.9 M 2024-09-18T01:05:49.453 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:49.700 INFO:teuthology.orchestra.run.smithi120.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 402 kB/s | 80 kB 00:00 2024-09-18T01:05:49.750 INFO:teuthology.orchestra.run.smithi120.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.5 MB/s | 897 kB 00:00 2024-09-18T01:05:49.751 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:49.751 INFO:teuthology.orchestra.run.smithi120.stdout:Total 3.2 MB/s | 977 kB 00:00 2024-09-18T01:05:49.763 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:49.784 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:49.784 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:49.864 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:49.865 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:50.039 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:50.178 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:50.179 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:50.179 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-18T01:05:50.179 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:50.179 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 285 k 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 1.3 M 2024-09-18T01:05:50.180 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:50.200 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T01:05:50.275 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:05:50.413 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-18T01:05:50.413 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:50.413 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-18T01:05:50.414 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:50.422 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:50.422 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:50.446 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:50.446 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:50.666 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:50.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:50.756 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:50.756 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:50.756 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T01:05:50.756 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:50.975 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:05:50.975 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-18T01:05:51.248 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T01:05:51.248 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:51.249 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:51.249 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T01:05:51.249 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:51.249 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:51.533 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-volume 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:51.756 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:52.053 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados-devel 2024-09-18T01:05:52.131 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:52.264 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:52.264 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 285 k 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 1.3 M 2024-09-18T01:05:52.265 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:52.491 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-18T01:05:52.491 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:52.492 INFO:teuthology.orchestra.run.smithi120.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-18T01:05:52.492 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:52.500 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:52.500 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:52.525 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:52.525 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:52.648 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:52.786 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:52.787 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-18T01:05:52.787 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:52.787 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 127 k 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 456 k 2024-09-18T01:05:52.788 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:52.826 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:05:52.826 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T01:05:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:52.994 INFO:teuthology.orchestra.run.smithi162.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 614 kB/s | 127 kB 00:00 2024-09-18T01:05:52.995 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:52.995 INFO:teuthology.orchestra.run.smithi162.stdout:Total 609 kB/s | 127 kB 00:00 2024-09-18T01:05:52.995 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:53.000 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:53.001 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:53.065 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:53.065 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:53.235 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:53.756 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:05:53.757 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:53.757 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:53.757 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:05:53.757 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:53.757 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:54.030 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:54.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install librados-devel 2024-09-18T01:05:54.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs2 2024-09-18T01:05:54.681 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:54.815 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:54.816 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:54.816 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repo Size 2024-09-18T01:05:54.816 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 127 k 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 456 k 2024-09-18T01:05:54.817 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:54.951 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:55.030 INFO:teuthology.orchestra.run.smithi120.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 594 kB/s | 127 kB 00:00 2024-09-18T01:05:55.031 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:55.031 INFO:teuthology.orchestra.run.smithi120.stdout:Total 590 kB/s | 127 kB 00:00 2024-09-18T01:05:55.031 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:55.037 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:55.037 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:55.045 INFO:teuthology.orchestra.run.smithi162.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:55.090 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:55.091 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:55.091 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:55.100 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:55.101 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:55.157 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs-devel 2024-09-18T01:05:55.247 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:55.345 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:55.738 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:55.748 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:55.870 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:55.871 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:55.871 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-18T01:05:55.871 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:55.871 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:05:55.871 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 32 k 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 142 k 2024-09-18T01:05:55.872 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:05:56.013 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:56.013 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:56.013 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:56.013 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:56.013 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:56.014 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:56.023 INFO:teuthology.orchestra.run.smithi162.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 209 kB/s | 32 kB 00:00 2024-09-18T01:05:56.024 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:56.024 INFO:teuthology.orchestra.run.smithi162.stdout:Total 208 kB/s | 32 kB 00:00 2024-09-18T01:05:56.024 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:05:56.028 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:05:56.028 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:05:56.044 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:05:56.045 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:05:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:05:56.246 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:56.282 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install libcephfs2 2024-09-18T01:05:56.633 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:56.876 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:05:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:05:56.877 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:56.878 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:56.972 INFO:teuthology.orchestra.run.smithi120.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:57.016 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:57.017 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:05:57.017 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:57.075 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados2 2024-09-18T01:05:57.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install libcephfs-devel 2024-09-18T01:05:57.653 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:57.670 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:57.741 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:57.785 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:57.786 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:57.786 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:57.801 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:57.802 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:57.802 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repo Size 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:05:57.803 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:05:57.804 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:05:57.804 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:57.804 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 32 k 2024-09-18T01:05:57.804 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 142 k 2024-09-18T01:05:57.804 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:05:57.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librbd1 2024-09-18T01:05:57.969 INFO:teuthology.orchestra.run.smithi120.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 191 kB/s | 32 kB 00:00 2024-09-18T01:05:57.969 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:05:57.970 INFO:teuthology.orchestra.run.smithi120.stdout:Total 190 kB/s | 32 kB 00:00 2024-09-18T01:05:57.970 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:05:57.973 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:05:57.973 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:05:57.990 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:05:57.990 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:05:58.080 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:05:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:58.437 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:58.527 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:58.547 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:58.571 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:58.571 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:58.572 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:58.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rados 2024-09-18T01:05:58.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:05:58.779 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:58.780 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:05:58.780 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:05:58.780 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:05:58.780 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:58.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install librados2 2024-09-18T01:05:59.224 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:59.314 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:59.357 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:05:59.358 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:05:59.358 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:05:59.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rgw 2024-09-18T01:05:59.543 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:05:59.631 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:05:59.674 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:05:59.675 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:05:59.675 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:05:59.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install librbd1 2024-09-18T01:06:00.020 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:00.111 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:00.154 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:00.155 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:06:00.155 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:00.225 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-cephfs 2024-09-18T01:06:00.313 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:00.402 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:00.446 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:00.447 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:06:00.447 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:00.509 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install python3-rados 2024-09-18T01:06:00.808 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:00.897 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:00.940 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:00.941 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:06:00.941 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:01.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rbd 2024-09-18T01:06:01.090 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:01.179 INFO:teuthology.orchestra.run.smithi120.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:01.222 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:01.223 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:06:01.223 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:01.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install python3-rgw 2024-09-18T01:06:01.606 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:01.696 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:01.739 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:01.740 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-18T01:06:01.740 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:01.814 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-fuse 2024-09-18T01:06:01.869 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:01.957 INFO:teuthology.orchestra.run.smithi120.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:02.001 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:02.002 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:06:02.002 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:02.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install python3-cephfs 2024-09-18T01:06:02.391 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:44 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:02.523 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:02.524 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 85 k 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 231 k 2024-09-18T01:06:02.525 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:06:02.649 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:44 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:02.718 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 442 kB/s | 85 kB 00:00 2024-09-18T01:06:02.719 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:02.719 INFO:teuthology.orchestra.run.smithi162.stdout:Total 439 kB/s | 85 kB 00:00 2024-09-18T01:06:02.719 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:06:02.726 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:06:02.726 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:06:02.736 INFO:teuthology.orchestra.run.smithi120.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:02.779 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:02.780 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:06:02.780 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:02.790 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:06:02.790 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:06:02.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install python3-rbd 2024-09-18T01:06:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:06:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:03.431 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:03.450 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:03.519 INFO:teuthology.orchestra.run.smithi120.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:06:03.562 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:03.563 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-09-18T01:06:03.563 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:03.628 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install rbd-fuse 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:03.718 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:03.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-mirror 2024-09-18T01:06:04.205 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:46 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:04.337 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:04.338 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:04.338 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-18T01:06:04.338 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:04.338 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:06:04.338 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 85 k 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 231 k 2024-09-18T01:06:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:06:04.506 INFO:teuthology.orchestra.run.smithi120.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 513 kB/s | 85 kB 00:00 2024-09-18T01:06:04.507 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:04.507 INFO:teuthology.orchestra.run.smithi120.stdout:Total 509 kB/s | 85 kB 00:00 2024-09-18T01:06:04.507 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:06:04.513 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:06:04.514 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:06:04.577 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:06:04.577 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:06:04.594 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:46 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:04.721 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:06:04.737 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:04.737 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:04.737 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 3.1 M 2024-09-18T01:06:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 13 M 2024-09-18T01:06:04.739 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:06:04.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:05.056 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.7 MB/s | 3.1 MB 00:00 2024-09-18T01:06:05.057 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:05.057 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.7 MB/s | 3.1 MB 00:00 2024-09-18T01:06:05.058 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:06:05.067 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:06:05.067 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:06:05.152 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:06:05.153 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:06:05.205 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:05.464 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:05.464 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:05.464 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:06:05.464 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:05.465 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:05.465 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:05.584 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:06:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:05.684 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:06:05.684 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T01:06:05.684 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:06:05.685 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:06:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:05.756 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install rbd-mirror 2024-09-18T01:06:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:48 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:06.506 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repo Size 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:06:06.507 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:06.508 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 3.1 M 2024-09-18T01:06:06.508 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 13 M 2024-09-18T01:06:06.508 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:06:06.820 INFO:teuthology.orchestra.run.smithi120.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 10 MB/s | 3.1 MB 00:00 2024-09-18T01:06:06.820 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:06.821 INFO:teuthology.orchestra.run.smithi120.stdout:Total 9.9 MB/s | 3.1 MB 00:00 2024-09-18T01:06:06.821 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:06:06.830 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:06:06.830 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:06.858 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:06.915 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:06:06.915 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:06:07.105 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-nbd 2024-09-18T01:06:07.359 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:06:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:07.464 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:06:07.464 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T01:06:07.464 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:06:07.465 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:06:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:07.698 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:49 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:07.837 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-18T01:06:07.838 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-18T01:06:07.839 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:07.839 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 170 k 2024-09-18T01:06:07.839 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 494 k 2024-09-18T01:06:07.839 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-18T01:06:08.053 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 796 kB/s | 170 kB 00:00 2024-09-18T01:06:08.054 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:08.054 INFO:teuthology.orchestra.run.smithi162.stdout:Total 790 kB/s | 170 kB 00:00 2024-09-18T01:06:08.055 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-18T01:06:08.062 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-18T01:06:08.062 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-18T01:06:08.126 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-18T01:06:08.126 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-18T01:06:08.278 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-18T01:06:08.384 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:08.589 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:08.805 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:08.806 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install rbd-nbd 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:09.079 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-18T01:06:09.350 DEBUG:teuthology.parallel:result is None 2024-09-18T01:06:09.393 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:51 ago on Wed 18 Sep 2024 01:04:18 AM UTC. 2024-09-18T01:06:09.529 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-18T01:06:09.530 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-18T01:06:09.530 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:09.530 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout:Install 1 Package 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 170 k 2024-09-18T01:06:09.531 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 494 k 2024-09-18T01:06:09.532 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-18T01:06:09.742 INFO:teuthology.orchestra.run.smithi120.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 805 kB/s | 170 kB 00:00 2024-09-18T01:06:09.743 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:06:09.743 INFO:teuthology.orchestra.run.smithi120.stdout:Total 799 kB/s | 170 kB 00:00 2024-09-18T01:06:09.743 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-18T01:06:09.751 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-18T01:06:09.751 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-18T01:06:09.814 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-18T01:06:09.815 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-18T01:06:09.967 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-18T01:06:10.073 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:06:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:10.732 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-18T01:06:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:06:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:10.733 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-18T01:06:10.961 DEBUG:teuthology.parallel:result is None 2024-09-18T01:06:10.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:06:11.106 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T01:06:11.127 INFO:teuthology.orchestra.run.smithi120.stdout:19.1.1-330.gc2ddf972.el9 2024-09-18T01:06:11.127 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-18T01:06:11.127 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-18T01:06:11.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:06:11.272 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T01:06:11.298 INFO:teuthology.orchestra.run.smithi162.stdout:19.1.1-330.gc2ddf972.el9 2024-09-18T01:06:11.298 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-18T01:06:11.298 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-18T01:06:11.300 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T01:06:11.300 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:11.300 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T01:06:11.325 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:11.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T01:06:11.367 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T01:06:11.368 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:11.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T01:06:11.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T01:06:11.459 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:11.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T01:06:11.488 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T01:06:11.555 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T01:06:11.555 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:11.555 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T01:06:11.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T01:06:11.644 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:11.644 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T01:06:11.671 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T01:06:11.739 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T01:06:11.739 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:11.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T01:06:11.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T01:06:11.834 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:11.834 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T01:06:11.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T01:06:11.926 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T01:06:12.070 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T01:06:12.071 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-09-18T01:06:12.073 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2024-09-18T01:06:12.099 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T01:06:12.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T01:06:12.138 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T01:06:12.165 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T01:06:12.165 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T01:06:12.203 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T01:06:12.239 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T01:06:12.240 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T01:06:12.240 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909695', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7WO3oM2Q4yjhxBxKd/6iJY3/m826ifegLDPxFP9LlVmTRKiDtFyiFws/mBMg60ETh3sG68xKTVYZRxp0kuj/g=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwWsyTgolgu0Fbg0FAUr/4DIEhKvWbgbf7nRhUSrBCat7yDPz8h2c0oS98nZel8UVHLJ9lhhUv1ogmHetTgvTc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'deep_copy': True, 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T01:06:12.240 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T01:06:12.267 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T01:06:12.285 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T01:06:12.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T01:06:12.350 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:12.350 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T01:06:12.402 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T01:06:12.402 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-18 01:06:07.713067459 +0000 2024-09-18T01:06:12.457 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-18 01:03:31.663935804 +0000 2024-09-18T01:06:12.458 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-18 01:03:31.663935804 +0000 2024-09-18T01:06:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-18 01:03:31.663935804 +0000 2024-09-18T01:06:12.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T01:06:12.519 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-18T01:06:12.519 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-18T01:06:12.519 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000202803 s, 2.5 MB/s 2024-09-18T01:06:12.520 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T01:06:12.577 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-18 01:06:07.713067459 +0000 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-18 01:03:31.992928346 +0000 2024-09-18T01:06:12.632 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-18 01:03:31.992928346 +0000 2024-09-18T01:06:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-18 01:03:31.992928346 +0000 2024-09-18T01:06:12.633 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T01:06:12.695 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-18T01:06:12.695 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-18T01:06:12.695 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000215476 s, 2.4 MB/s 2024-09-18T01:06:12.696 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T01:06:12.753 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-18 01:06:07.713067459 +0000 2024-09-18T01:06:12.808 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-18 01:03:32.313921070 +0000 2024-09-18T01:06:12.809 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-18 01:03:32.313921070 +0000 2024-09-18T01:06:12.809 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-18 01:03:32.313921070 +0000 2024-09-18T01:06:12.809 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T01:06:12.871 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-18T01:06:12.871 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-18T01:06:12.871 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000305873 s, 1.7 MB/s 2024-09-18T01:06:12.873 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T01:06:12.929 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-18 01:06:07.714067436 +0000 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-18 01:03:32.635913770 +0000 2024-09-18T01:06:12.985 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-18 01:03:32.635913770 +0000 2024-09-18T01:06:12.986 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-18 01:03:32.635913770 +0000 2024-09-18T01:06:12.986 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T01:06:13.048 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-18T01:06:13.048 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-18T01:06:13.048 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000158577 s, 3.2 MB/s 2024-09-18T01:06:13.049 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T01:06:13.105 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:13.105 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:13.105 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T01:06:13.122 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T01:06:13.122 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:13.180 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-18 01:06:05.941423650 +0000 2024-09-18T01:06:13.181 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-18 01:03:34.407085948 +0000 2024-09-18T01:06:13.181 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-18 01:03:34.407085948 +0000 2024-09-18T01:06:13.181 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-18 01:03:34.407085948 +0000 2024-09-18T01:06:13.181 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T01:06:13.245 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-18T01:06:13.245 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-18T01:06:13.245 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000249297 s, 2.1 MB/s 2024-09-18T01:06:13.246 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T01:06:13.304 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-18 01:06:05.942423631 +0000 2024-09-18T01:06:13.360 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-18 01:03:34.730079588 +0000 2024-09-18T01:06:13.361 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-18 01:03:34.730079588 +0000 2024-09-18T01:06:13.361 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-18 01:03:34.730079588 +0000 2024-09-18T01:06:13.361 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T01:06:13.424 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-18T01:06:13.424 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-18T01:06:13.424 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000194247 s, 2.6 MB/s 2024-09-18T01:06:13.425 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T01:06:13.482 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-18 01:06:05.942423631 +0000 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-18 01:03:35.075072795 +0000 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-18 01:03:35.075072795 +0000 2024-09-18T01:06:13.538 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-18 01:03:35.075072795 +0000 2024-09-18T01:06:13.539 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T01:06:13.601 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-18T01:06:13.601 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-18T01:06:13.601 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000226107 s, 2.3 MB/s 2024-09-18T01:06:13.603 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T01:06:13.659 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-09-18T01:06:13.714 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T01:06:13.714 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:06:13.714 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-18T01:06:13.714 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:06:13.714 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:06:13.715 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-18 01:06:05.942423631 +0000 2024-09-18T01:06:13.715 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-18 01:03:35.430065805 +0000 2024-09-18T01:06:13.715 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-18 01:03:35.430065805 +0000 2024-09-18T01:06:13.715 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-18 01:03:35.430065805 +0000 2024-09-18T01:06:13.715 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T01:06:13.777 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-18T01:06:13.777 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-18T01:06:13.777 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000187757 s, 2.7 MB/s 2024-09-18T01:06:13.779 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T01:06:13.835 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:13.835 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T01:06:13.835 INFO:tasks.ceph:Generating config... 2024-09-18T01:06:13.836 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T01:06:13.836 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-18T01:06:13.837 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T01:06:13.837 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T01:06:13.837 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T01:06:13.837 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T01:06:13.837 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T01:06:13.837 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-18T01:06:13.837 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T01:06:13.838 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T01:06:13.838 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T01:06:13.839 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T01:06:13.888 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T01:06:13.891 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T01:06:13.935 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:06:13.999 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.120'), ('mon.c', '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]'), ('mon.b', '172.21.15.162')] 2024-09-18T01:06:14.000 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790],172.21.15.162', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T01:06:14.001 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:14.001 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T01:06:14.055 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.120 --addv c '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]' --add b 172.21.15.162 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: generated fsid e017e0c5-d4e1-4f77-8127-47c02525e65d 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:setting min_mon_release = quincy 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 0 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:fsid e017e0c5-d4e1-4f77-8127-47c02525e65d 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:last_changed 2024-09-18T01:06:14.140651+0000 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-18T01:06:14.140651+0000 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:min_mon_release 17 (quincy) 2024-09-18T01:06:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:election_strategy: 1 2024-09-18T01:06:14.142 INFO:teuthology.orchestra.run.smithi120.stdout:0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-18T01:06:14.142 INFO:teuthology.orchestra.run.smithi120.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.b 2024-09-18T01:06:14.142 INFO:teuthology.orchestra.run.smithi120.stdout:2: [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] mon.c 2024-09-18T01:06:14.142 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T01:06:14.143 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T01:06:14.158 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e017e0c5-d4e1-4f77-8127-47c02525e65d... 2024-09-18T01:06:14.158 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T01:06:14.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T01:06:14.240 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-09-18T01:06:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-09-18T01:06:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T01:06:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T01:06:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-09-18T01:06:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-09-18T01:06:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790],172.21.15.162" 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: mon client directed command retry = 5 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on pool no app = False 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: ms inject socket failures = 5000 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = e017e0c5-d4e1-4f77-8127-47c02525e65d 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 600 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-09-18T01:06:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bdev async discard = True 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bdev enable discard = True 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore allocator = stupid 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-09-18T01:06:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 1/20 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 1/20 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 4/10 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: osd blocked scrub grace period = 3600 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: osd max markdown count = 1000 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: osd shutdown pgref assert = True 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.247 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-09-18T01:06:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T01:06:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-09-18T01:06:14.250 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-09-18T01:06:14.251 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-09-18T01:06:14.252 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T01:06:14.253 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790],172.21.15.162" 2024-09-18T01:06:14.254 INFO:teuthology.orchestra.run.smithi162.stdout: mon client directed command retry = 5 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on pool no app = False 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject socket failures = 5000 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = e017e0c5-d4e1-4f77-8127-47c02525e65d 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-09-18T01:06:14.255 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-09-18T01:06:14.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bdev async discard = True 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bdev enable discard = True 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore allocator = stupid 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 1/20 2024-09-18T01:06:14.257 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 1/20 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 4/10 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: osd blocked scrub grace period = 3600 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: osd max markdown count = 1000 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: osd shutdown pgref assert = True 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-09-18T01:06:14.258 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-09-18T01:06:14.259 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.260 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-09-18T01:06:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-09-18T01:06:14.263 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T01:06:14.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T01:06:14.357 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T01:06:14.358 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:14.358 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T01:06:14.373 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:14.373 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T01:06:14.430 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:06:14.430 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:14.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T01:06:14.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:06:14.504 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:14.534 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:14.560 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:06:14.560 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:14.560 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T01:06:14.561 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:06:14.596 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:14.596 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:14.651 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T01:06:14.652 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T01:06:14.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T01:06:14.702 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T01:06:14.705 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T01:06:14.763 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T01:06:14.765 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T01:06:14.766 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T01:06:14.766 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T01:06:14.806 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T01:06:14.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T01:06:14.860 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T01:06:14.873 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T01:06:14.873 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T01:06:14.873 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T01:06:14.897 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:14.897 INFO:tasks.ceph:role: osd.0 2024-09-18T01:06:14.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:06:14.898 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T01:06:14.970 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:14.970 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:14.970 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:14.970 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:14.970 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:14.971 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:14.971 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:14.971 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:14.971 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:14.971 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:15.466 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-18T01:06:15.468 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-18T01:06:15.468 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T01:06:15.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T01:06:15.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T01:06:15.701 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:15.702 INFO:tasks.ceph:role: osd.1 2024-09-18T01:06:15.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:06:15.702 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:15.774 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:15.775 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:16.227 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-18T01:06:16.229 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-18T01:06:16.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T01:06:16.316 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T01:06:16.346 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T01:06:16.410 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:16.410 INFO:tasks.ceph:role: osd.2 2024-09-18T01:06:16.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:06:16.411 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T01:06:16.483 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:16.484 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:16.971 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-18T01:06:16.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-18T01:06:16.973 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T01:06:17.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T01:06:17.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T01:06:17.180 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:17.180 INFO:tasks.ceph:role: osd.3 2024-09-18T01:06:17.180 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:06:17.180 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T01:06:17.252 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:17.253 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:17.765 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-18T01:06:17.767 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-18T01:06:17.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T01:06:17.917 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T01:06:17.947 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:18.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:18.068+0000 7f3241b19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T01:06:18.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:18.069+0000 7f3241b19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T01:06:18.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:18.069+0000 7f3241b19640 -1 bdev(0x557644795800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:18.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:18.069+0000 7f3241b19640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T01:06:21.969 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T01:06:21.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:22.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:22.087+0000 7f3a38c9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T01:06:22.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:22.087+0000 7f3a38c9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T01:06:22.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:22.087+0000 7f3a38c9c640 -1 bdev(0x55dd9f6c7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:22.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:22.088+0000 7f3a38c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T01:06:25.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T01:06:25.789 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:25.876+0000 7fac1240f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T01:06:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:25.876+0000 7fac1240f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T01:06:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:25.876+0000 7fac1240f640 -1 bdev(0x559f23077800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:25.876+0000 7fac1240f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T01:06:29.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T01:06:29.814 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:29.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:29.904+0000 7fe39b80d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T01:06:29.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:29.904+0000 7fe39b80d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T01:06:29.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:29.904+0000 7fe39b80d640 -1 bdev(0x5609aa125800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:29.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:06:29.904+0000 7fe39b80d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T01:06:33.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T01:06:33.836 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T01:06:33.867 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:33.867 INFO:tasks.ceph:role: osd.4 2024-09-18T01:06:33.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:06:33.867 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T01:06:33.944 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:33.944 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:33.944 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:33.944 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:33.944 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:33.945 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:33.945 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:34.445 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-18T01:06:34.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-18T01:06:34.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T01:06:34.554 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T01:06:34.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T01:06:34.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:34.652 INFO:tasks.ceph:role: osd.5 2024-09-18T01:06:34.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:06:34.652 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:34.725 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:34.726 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:34.726 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:34.726 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:35.176 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-18T01:06:35.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-18T01:06:35.178 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T01:06:35.250 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T01:06:35.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T01:06:35.387 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:35.388 INFO:tasks.ceph:role: osd.6 2024-09-18T01:06:35.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:06:35.388 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T01:06:35.460 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:35.460 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:35.460 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:35.460 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:35.460 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:35.461 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:35.461 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:35.461 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:35.461 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:35.461 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:35.945 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-18T01:06:35.947 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-18T01:06:35.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T01:06:36.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T01:06:36.085 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T01:06:36.150 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T01:06:36.151 INFO:tasks.ceph:role: osd.7 2024-09-18T01:06:36.151 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:06:36.151 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T01:06:36.224 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:06:36.225 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:06:36.726 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-18T01:06:36.728 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-18T01:06:36.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T01:06:36.841 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T01:06:36.871 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:36.971+0000 7f5838211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T01:06:36.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:36.976+0000 7f5838211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T01:06:36.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:36.976+0000 7f5838211640 -1 bdev(0x55ee4c195800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:36.977 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:36.976+0000 7f5838211640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T01:06:40.641 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T01:06:40.671 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:40.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:40.761+0000 7ff2e5218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T01:06:40.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:40.761+0000 7ff2e5218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T01:06:40.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:40.761+0000 7ff2e5218640 -1 bdev(0x5641c69bd800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:40.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:40.762+0000 7ff2e5218640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T01:06:44.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T01:06:44.690 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:44.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:44.779+0000 7f6247c20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T01:06:44.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:44.779+0000 7f6247c20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T01:06:44.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:44.779+0000 7f6247c20640 -1 bdev(0x55bcc7fd3800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:44.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:44.779+0000 7f6247c20640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T01:06:48.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T01:06:48.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:48.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:48.812+0000 7fdde5c2a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T01:06:48.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:48.813+0000 7fdde5c2a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T01:06:48.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:48.813+0000 7fdde5c2a640 -1 bdev(0x55bfc122b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-18T01:06:48.814 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-18T01:06:48.813+0000 7fdde5c2a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T01:06:52.476 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T01:06:52.505 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T01:06:52.505 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:52.506 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T01:06:52.536 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:52.536 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T01:06:52.605 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:52.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T01:06:52.672 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:52.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T01:06:52.739 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:52.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T01:06:52.806 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:52.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T01:06:52.836 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:52.836 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T01:06:52.905 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:52.905 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T01:06:52.972 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:52.973 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T01:06:53.039 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:53.040 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T01:06:53.106 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:53.106 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T01:06:53.123 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:06:53.123 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T01:06:53.162 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T01:06:53.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T01:06:53.165 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T01:06:53.205 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCGJ+pml3rfKRAAXxjLZalaTzApeDmypPk+xg== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCKJ+pmfAA0BBAA4WHC0PbZNdQhSXBBiag1WQ== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCOJ+pmSmpUBRAA64nqszkTKqwHQ15HaBL2fg== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCRJ+pmJqtHNBAA4hTpJG91ky5Jxoc4l6aJNg== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCVJ+pmAv30NRAA8KH4tx5rEmfzst411tBn5A== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCGJ+pmjsV4LRAAL9EyYloOCWNBYsediFj7uw== 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-09-18T01:06:53.206 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCcJ+pmO4j7ORAAqpAs0LhwuSLGcjQE3PStkw== 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCgJ+pmmoNzLRAAAzShUWH09brfbmL5unax4A== 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCkJ+pmdgqFLhAAzgsTcsBjVprO0x9Wtycxow== 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCoJ+pmGIuCMBAATpsW64nxIhvstuYjva8G1g== 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCGJ+pm5RUVMBAAcXF3RmddccaVfdICDmjU1g== 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout:[client.1] 2024-09-18T01:06:53.207 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCGJ+pmy9hBMxAA10qSR5QCtCs4kKW/ouHlow== 2024-09-18T01:06:53.226 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2024-09-18T01:06:53.226 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCGJ+pml3rfKRAAXxjLZalaTzApeDmypPk+xg== 2024-09-18T01:06:53.226 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2024-09-18T01:06:53.226 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCKJ+pmfAA0BBAA4WHC0PbZNdQhSXBBiag1WQ== 2024-09-18T01:06:53.226 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCOJ+pmSmpUBRAA64nqszkTKqwHQ15HaBL2fg== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCRJ+pmJqtHNBAA4hTpJG91ky5Jxoc4l6aJNg== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCVJ+pmAv30NRAA8KH4tx5rEmfzst411tBn5A== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCGJ+pmjsV4LRAAL9EyYloOCWNBYsediFj7uw== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCcJ+pmO4j7ORAAqpAs0LhwuSLGcjQE3PStkw== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCgJ+pmmoNzLRAAAzShUWH09brfbmL5unax4A== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCkJ+pmdgqFLhAAzgsTcsBjVprO0x9Wtycxow== 2024-09-18T01:06:53.227 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2024-09-18T01:06:53.228 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCoJ+pmGIuCMBAATpsW64nxIhvstuYjva8G1g== 2024-09-18T01:06:53.228 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-09-18T01:06:53.228 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCGJ+pm5RUVMBAAcXF3RmddccaVfdICDmjU1g== 2024-09-18T01:06:53.228 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2024-09-18T01:06:53.228 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCGJ+pmy9hBMxAA10qSR5QCtCs4kKW/ouHlow== 2024-09-18T01:06:53.229 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:06:53.250 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:06:53.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.324 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.428 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.535 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:06:53.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:06:53.588 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.695 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:06:53.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:06:53.801 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:06:53.855 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:06:53.857 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:06:53.911 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T01:06:53.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T01:06:53.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:06:54.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T01:06:54.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T01:06:54.122 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:06:54.217 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T01:06:54.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T01:06:54.267 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:06:54.363 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T01:06:54.388 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:54.390 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:06:54.443 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T01:06:54.444 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T01:06:54.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T01:06:54.446 INFO:tasks.ceph.mon.a:Started 2024-09-18T01:06:54.446 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T01:06:54.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T01:06:54.448 INFO:tasks.ceph.mon.c:Started 2024-09-18T01:06:54.448 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T01:06:54.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T01:06:54.485 INFO:tasks.ceph.mon.b:Started 2024-09-18T01:06:54.486 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T01:06:54.486 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T01:06:54.486 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T01:06:54.488 INFO:tasks.ceph.mgr.y:Started 2024-09-18T01:06:54.488 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T01:06:54.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T01:06:54.490 INFO:tasks.ceph.mgr.x:Started 2024-09-18T01:06:54.490 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T01:06:54.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T01:06:59.895 INFO:teuthology.orchestra.run.smithi120.stderr:adjusted tunables profile to default 2024-09-18T01:06:59.911 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T01:06:59.911 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T01:06:59.911 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:59.911 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T01:06:59.921 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:06:59.919+0000 7f4baa323040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T01:06:59.936 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:59.936 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T01:06:59.963 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:06:59.961+0000 7f4baa323040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T01:06:59.987 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:06:59.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T01:07:00.052 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-18T01:07:00.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T01:07:00.111 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.108+0000 7f4baa323040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T01:07:00.117 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:07:00.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T01:07:00.148 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:07:00.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T01:07:00.161 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.159+0000 7f4baa323040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T01:07:00.217 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:07:00.217 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T01:07:00.285 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-18T01:07:00.285 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T01:07:00.324 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.322+0000 7f4baa323040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:07:00.352 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 13ff2fc9-b863-45cb-b111-62d2ff06dff1 0 2024-09-18T01:07:00.431 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.429+0000 7f4baa323040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T01:07:00.575 INFO:teuthology.orchestra.run.smithi162.stdout:0 2024-09-18T01:07:00.589 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 01867b0a-aeb8-4b92-897b-f8176abf95a9 1 2024-09-18T01:07:00.668 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:00.667+0000 7fe042b18040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T01:07:00.712 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:00.710+0000 7fe042b18040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T01:07:00.772 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.769+0000 7f4baa323040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T01:07:00.773 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-09-18T01:07:00.785 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 788e3030-dfc0-45b5-9c99-763dc9283283 2 2024-09-18T01:07:00.821 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.819+0000 7f4baa323040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T01:07:00.862 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:00.860+0000 7fe042b18040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T01:07:00.926 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:00.925+0000 7fe042b18040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T01:07:00.969 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:00.967+0000 7f4baa323040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T01:07:00.983 INFO:teuthology.orchestra.run.smithi162.stdout:2 2024-09-18T01:07:00.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 9ecd2117-0d0b-4629-9e54-3578a68fa0db 3 2024-09-18T01:07:01.122 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:01.121+0000 7fe042b18040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:07:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:3 2024-09-18T01:07:01.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 3af2d4ee-bce6-49df-a10e-f7c367d9fcb5 4 2024-09-18T01:07:01.239 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:01.237+0000 7fe042b18040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T01:07:01.374 INFO:teuthology.orchestra.run.smithi162.stdout:4 2024-09-18T01:07:01.385 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new fbac8f09-7655-4b5e-8bb5-74650aa3c2f1 5 2024-09-18T01:07:01.571 INFO:teuthology.orchestra.run.smithi162.stdout:5 2024-09-18T01:07:01.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 1a98a211-cab7-414a-ad17-b06cede06bcf 6 2024-09-18T01:07:01.583 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:01.582+0000 7fe042b18040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T01:07:01.641 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:01.640+0000 7fe042b18040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T01:07:01.765 INFO:teuthology.orchestra.run.smithi162.stdout:6 2024-09-18T01:07:01.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 494f3743-c1f1-410f-b5f8-ce5927494495 7 2024-09-18T01:07:01.801 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:01.800+0000 7fe042b18040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T01:07:01.947 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:01.945+0000 7f4baa323040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T01:07:01.956 INFO:teuthology.orchestra.run.smithi162.stdout:7 2024-09-18T01:07:01.967 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T01:07:01.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T01:07:01.969 INFO:tasks.ceph.osd.0:Started 2024-09-18T01:07:01.969 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T01:07:01.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T01:07:01.972 INFO:tasks.ceph.osd.1:Started 2024-09-18T01:07:01.972 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T01:07:01.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T01:07:01.974 INFO:tasks.ceph.osd.2:Started 2024-09-18T01:07:01.975 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T01:07:01.975 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T01:07:01.977 INFO:tasks.ceph.osd.3:Started 2024-09-18T01:07:01.977 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T01:07:01.977 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T01:07:02.009 INFO:tasks.ceph.osd.4:Started 2024-09-18T01:07:02.009 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T01:07:02.010 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T01:07:02.011 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.009+0000 7f4baa323040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T01:07:02.011 INFO:tasks.ceph.osd.5:Started 2024-09-18T01:07:02.011 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T01:07:02.012 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T01:07:02.013 INFO:tasks.ceph.osd.6:Started 2024-09-18T01:07:02.014 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T01:07:02.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T01:07:02.016 INFO:tasks.ceph.osd.7:Started 2024-09-18T01:07:02.016 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:02.198 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:02.198 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:01.953049+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:02.209 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T01:07:02.209 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T01:07:02.406 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.404+0000 7f4baa323040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T01:07:02.489 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.486+0000 7f4baa323040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T01:07:02.594 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.592+0000 7f4baa323040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:07:02.641 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.639+0000 7f4baa323040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T01:07:02.755 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.752+0000 7f4baa323040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T01:07:02.808 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.806+0000 7f4baa323040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T01:07:02.814 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:02.813+0000 7fe042b18040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T01:07:02.852 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.850+0000 7f4baa323040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T01:07:02.866 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:02.865+0000 7fe042b18040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T01:07:02.966 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:02.964+0000 7f4baa323040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T01:07:03.017 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:03.018 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.012+0000 7f4baa323040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T01:07:03.068 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.066+0000 7f4baa323040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T01:07:03.155 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.153+0000 7f4baa323040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T01:07:03.205 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-18T01:07:03.206 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":10,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:01.953049+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:03.208 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.206+0000 7f4baa323040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T01:07:03.215 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T01:07:03.233 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.231+0000 7fe042b18040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T01:07:03.254 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.252+0000 7f4baa323040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T01:07:03.319 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.318+0000 7fe042b18040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T01:07:03.426 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.425+0000 7fe042b18040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:07:03.444 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:03.442+0000 7f45a881f640 -1 Falling back to public interface 2024-09-18T01:07:03.456 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:03.454+0000 7fae0592c640 -1 Falling back to public interface 2024-09-18T01:07:03.472 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.471+0000 7fe042b18040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T01:07:03.533 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:03.531+0000 7f541aedc640 -1 Falling back to public interface 2024-09-18T01:07:03.588 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.587+0000 7fe042b18040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T01:07:03.641 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.640+0000 7fe042b18040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T01:07:03.681 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:03.678+0000 7f6781d34640 -1 Falling back to public interface 2024-09-18T01:07:03.686 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.684+0000 7fe042b18040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T01:07:03.754 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:03.752+0000 7f7bb0812640 -1 Falling back to public interface 2024-09-18T01:07:03.760 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:03.759+0000 7fa07cda4640 -1 Falling back to public interface 2024-09-18T01:07:03.761 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:03.759+0000 7f8042826640 -1 Falling back to public interface 2024-09-18T01:07:03.761 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:03.760+0000 7f80417ce640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-09-18T01:07:03.801 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.800+0000 7fe042b18040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T01:07:03.852 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.850+0000 7fe042b18040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T01:07:03.903 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.902+0000 7fe042b18040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T01:07:03.909 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-18T01:07:03.907+0000 7f4baa323040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T01:07:03.934 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:03.932+0000 7fd7814f1640 -1 Falling back to public interface 2024-09-18T01:07:03.970 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:03.969+0000 7fe042b18040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T01:07:04.015 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:04.013+0000 7fe042b18040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T01:07:04.060 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:04.058+0000 7fe042b18040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T01:07:04.713 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:04.711+0000 7fe042b18040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T01:07:05.725 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:05.724+0000 7fdfe33e7640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T01:07:05.725 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-18T01:07:05.724+0000 7fdfe33e7640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T01:07:07.503 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:07.502+0000 7f45a881f640 -1 osd.3 0 log_to_monitors true 2024-09-18T01:07:07.565 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:07.564+0000 7fa07cda4640 -1 osd.4 0 log_to_monitors true 2024-09-18T01:07:07.785 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:07.784+0000 7fae0592c640 -1 osd.2 0 log_to_monitors true 2024-09-18T01:07:07.882 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:07.881+0000 7f541aedc640 -1 osd.7 0 log_to_monitors true 2024-09-18T01:07:08.117 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:08.116+0000 7f7bb0812640 -1 osd.6 0 log_to_monitors true 2024-09-18T01:07:08.140 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:08.139+0000 7f8042826640 -1 osd.5 0 log_to_monitors true 2024-09-18T01:07:08.205 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:08.204+0000 7f6781d34640 -1 osd.1 0 log_to_monitors true 2024-09-18T01:07:08.435 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:08.434+0000 7fd7814f1640 -1 osd.0 0 log_to_monitors true 2024-09-18T01:07:09.640 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:09.639+0000 7f45a4a00640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T01:07:09.657 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:09.656+0000 7f459bffb640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:09.671 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:09.669+0000 7fa078510640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T01:07:09.684 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:09.683+0000 7fa070b42640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:10.022 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:10.335 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-18T01:07:10.336 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":13,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:09.729651+0000","last_up_change":"2024-09-18T01:07:09.729651+0000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:10.347 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-18T01:07:11.527 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:11.526+0000 7f7bab87b640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T01:07:11.527 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:11.526+0000 7f677d4a0640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T01:07:11.532 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:11.531+0000 7fd77cc5d640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T01:07:11.542 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:11.541+0000 7f7ba3ead640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:11.544 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:11.543+0000 7f803df92640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T01:07:11.547 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:11.546+0000 7f6775ad2640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:11.547 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:11.547+0000 7fd77528f640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:11.561 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:11.560+0000 7f80365c4640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:11.570 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:11.569+0000 7fae01098640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T01:07:11.576 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:11.575+0000 7f5416648640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T01:07:11.588 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:11.587+0000 7fadf96ca640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:11.595 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:11.594+0000 7f540ec7a640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:07:17.153 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:17.433 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-18T01:07:17.433 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":17,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:13.897348+0000","last_up_change":"2024-09-18T01:07:11.886968+0000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6809","nonce":4172964327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6811","nonce":4172964327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6815","nonce":4172964327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6813","nonce":4172964327}]},"public_addr":"172.21.15.162:6809/4172964327","cluster_addr":"172.21.15.162:6811/4172964327","heartbeat_back_addr":"172.21.15.162:6815/4172964327","heartbeat_front_addr":"172.21.15.162:6813/4172964327","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:17.444 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T01:07:17.444 INFO:tasks.ceph:Creating RBD pool 2024-09-18T01:07:17.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T01:07:17.922 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'rbd' created 2024-09-18T01:07:17.935 DEBUG:teuthology.orchestra.run.smithi120:> rbd --cluster ceph pool init rbd 2024-09-18T01:07:20.960 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T01:07:20.961 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T01:07:20.961 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T01:07:20.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T01:07:20.962 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T01:07:21.296 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:21.307 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":5,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1958939040},{"type":"v1","addr":"172.21.15.162:6833","nonce":1958939040}]},"active_addr":"172.21.15.162:6833/1958939040","active_change":"2024-09-18T01:07:04.714742+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3925698288}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":858475858}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3201490137}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2009350411}]}]} 2024-09-18T01:07:21.311 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T01:07:21.311 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T01:07:21.311 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:21.590 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":21,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:20.935126+0000","last_up_change":"2024-09-18T01:07:11.886968+0000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:07:17.736243+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6809","nonce":4172964327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6811","nonce":4172964327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6815","nonce":4172964327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6813","nonce":4172964327}]},"public_addr":"172.21.15.162:6809/4172964327","cluster_addr":"172.21.15.162:6811/4172964327","heartbeat_back_addr":"172.21.15.162:6815/4172964327","heartbeat_front_addr":"172.21.15.162:6813/4172964327","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:21.602 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T01:07:21.603 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:07:21.879 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:21.896 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":21,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:07:20.935126+0000","last_up_change":"2024-09-18T01:07:11.886968+0000","last_in_change":"2024-09-18T01:07:01.953049+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:07:17.736243+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6809","nonce":4172964327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6811","nonce":4172964327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6815","nonce":4172964327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4172964327},{"type":"v1","addr":"172.21.15.162:6813","nonce":4172964327}]},"public_addr":"172.21.15.162:6809/4172964327","cluster_addr":"172.21.15.162:6811/4172964327","heartbeat_back_addr":"172.21.15.162:6815/4172964327","heartbeat_front_addr":"172.21.15.162:6813/4172964327","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:07:21.898 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T01:07:21.898 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T01:07:21.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T01:07:21.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T01:07:21.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T01:07:21.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T01:07:21.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T01:07:21.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T01:07:22.134 INFO:teuthology.orchestra.run.smithi120.stdout:55834574853 2024-09-18T01:07:22.135 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:07:22.140 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:22.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:07:22.142 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:22.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:07:22.143 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:22.144 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:07:22.145 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:22.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:07:22.147 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:22.147 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:07:22.148 INFO:teuthology.orchestra.run.smithi120.stdout:55834574854 2024-09-18T01:07:22.149 INFO:teuthology.orchestra.run.smithi120.stdout:64424509446 2024-09-18T01:07:22.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:07:22.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:07:22.699 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:22.701 INFO:teuthology.orchestra.run.smithi120.stdout:55834574852 2024-09-18T01:07:22.711 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:22.714 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:22.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-18T01:07:22.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-18T01:07:22.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-18T01:07:22.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.0 2024-09-18T01:07:22.746 INFO:teuthology.orchestra.run.smithi120.stdout:64424509443 2024-09-18T01:07:22.751 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:22.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.1 2024-09-18T01:07:22.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-09-18T01:07:22.776 INFO:teuthology.orchestra.run.smithi120.stdout:55834574852 2024-09-18T01:07:22.789 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:22.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.4 2024-09-18T01:07:22.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.2 2024-09-18T01:07:23.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:07:23.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:07:23.729 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:07:23.737 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:07:23.760 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:07:23.766 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:07:23.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:07:23.803 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:07:24.275 INFO:teuthology.orchestra.run.smithi120.stdout:55834574853 2024-09-18T01:07:24.288 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:24.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-18T01:07:24.296 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-09-18T01:07:24.307 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.335 INFO:teuthology.orchestra.run.smithi120.stdout:64424509444 2024-09-18T01:07:24.339 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:24.343 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:24.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.1 2024-09-18T01:07:24.351 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2024-09-18T01:07:24.354 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.361 INFO:teuthology.orchestra.run.smithi120.stdout:64424509446 2024-09-18T01:07:24.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-18T01:07:24.363 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-18T01:07:24.376 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.376 INFO:teuthology.orchestra.run.smithi120.stdout:64424509445 2024-09-18T01:07:24.383 INFO:teuthology.orchestra.run.smithi120.stdout:55834574854 2024-09-18T01:07:24.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-09-18T01:07:24.389 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-09-18T01:07:24.396 DEBUG:teuthology.parallel:result is None 2024-09-18T01:07:24.396 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:07:24.396 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:24.712 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:24.712 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:24.723 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-18T01:07:24.530686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219036,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2842624,"data_stored":1737870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.808833"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.030442+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:21.030442+0000","last_peered":"2024-09-18T01:07:21.030442+0000","last_clean":"2024-09-18T01:07:21.030442+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:21.030442+0000","last_undegraded":"2024-09-18T01:07:21.030442+0000","last_fullsized":"2024-09-18T01:07:21.030442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954875+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:20.954875+0000","last_peered":"2024-09-18T01:07:20.954875+0000","last_clean":"2024-09-18T01:07:20.954875+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:20.954875+0000","last_undegraded":"2024-09-18T01:07:20.954875+0000","last_fullsized":"2024-09-18T01:07:20.954875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867669+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:21.867669+0000","last_peered":"2024-09-18T01:07:21.867669+0000","last_clean":"2024-09-18T01:07:21.867669+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:21.867669+0000","last_undegraded":"2024-09-18T01:07:21.867669+0000","last_fullsized":"2024-09-18T01:07:21.867669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954917+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:20.954917+0000","last_peered":"2024-09-18T01:07:20.954917+0000","last_clean":"2024-09-18T01:07:20.954917+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:20.954917+0000","last_undegraded":"2024-09-18T01:07:20.954917+0000","last_fullsized":"2024-09-18T01:07:20.954917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.953958+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:20.953958+0000","last_peered":"2024-09-18T01:07:20.953958+0000","last_clean":"2024-09-18T01:07:20.953958+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:20.953958+0000","last_undegraded":"2024-09-18T01:07:20.953958+0000","last_fullsized":"2024-09-18T01:07:20.953958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.949080+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:20.949080+0000","last_peered":"2024-09-18T01:07:20.949080+0000","last_clean":"2024-09-18T01:07:20.949080+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:20.949080+0000","last_undegraded":"2024-09-18T01:07:20.949080+0000","last_fullsized":"2024-09-18T01:07:20.949080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867555+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:21.867555+0000","last_peered":"2024-09-18T01:07:21.867555+0000","last_clean":"2024-09-18T01:07:21.867555+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:21.867555+0000","last_undegraded":"2024-09-18T01:07:21.867555+0000","last_fullsized":"2024-09-18T01:07:21.867555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.952512+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:20.952512+0000","last_peered":"2024-09-18T01:07:20.952512+0000","last_clean":"2024-09-18T01:07:20.952512+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:20.952512+0000","last_undegraded":"2024-09-18T01:07:20.952512+0000","last_fullsized":"2024-09-18T01:07:20.952512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867098+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:21.867098+0000","last_peered":"2024-09-18T01:07:21.867098+0000","last_clean":"2024-09-18T01:07:21.867098+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:21.867098+0000","last_undegraded":"2024-09-18T01:07:21.867098+0000","last_fullsized":"2024-09-18T01:07:21.867098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:24.724 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:24.998 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:25.012 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T01:07:24.723041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219036,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2842624,"data_stored":1737870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.030442+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:21.030442+0000","last_peered":"2024-09-18T01:07:21.030442+0000","last_clean":"2024-09-18T01:07:21.030442+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:21.030442+0000","last_undegraded":"2024-09-18T01:07:21.030442+0000","last_fullsized":"2024-09-18T01:07:21.030442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954875+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:20.954875+0000","last_peered":"2024-09-18T01:07:20.954875+0000","last_clean":"2024-09-18T01:07:20.954875+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:20.954875+0000","last_undegraded":"2024-09-18T01:07:20.954875+0000","last_fullsized":"2024-09-18T01:07:20.954875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867669+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:21.867669+0000","last_peered":"2024-09-18T01:07:21.867669+0000","last_clean":"2024-09-18T01:07:21.867669+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:21.867669+0000","last_undegraded":"2024-09-18T01:07:21.867669+0000","last_fullsized":"2024-09-18T01:07:21.867669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954917+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:20.954917+0000","last_peered":"2024-09-18T01:07:20.954917+0000","last_clean":"2024-09-18T01:07:20.954917+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:20.954917+0000","last_undegraded":"2024-09-18T01:07:20.954917+0000","last_fullsized":"2024-09-18T01:07:20.954917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.953958+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:20.953958+0000","last_peered":"2024-09-18T01:07:20.953958+0000","last_clean":"2024-09-18T01:07:20.953958+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:20.953958+0000","last_undegraded":"2024-09-18T01:07:20.953958+0000","last_fullsized":"2024-09-18T01:07:20.953958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.949080+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:20.949080+0000","last_peered":"2024-09-18T01:07:20.949080+0000","last_clean":"2024-09-18T01:07:20.949080+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:20.949080+0000","last_undegraded":"2024-09-18T01:07:20.949080+0000","last_fullsized":"2024-09-18T01:07:20.949080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867555+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:21.867555+0000","last_peered":"2024-09-18T01:07:21.867555+0000","last_clean":"2024-09-18T01:07:21.867555+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:21.867555+0000","last_undegraded":"2024-09-18T01:07:21.867555+0000","last_fullsized":"2024-09-18T01:07:21.867555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.952512+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:20.952512+0000","last_peered":"2024-09-18T01:07:20.952512+0000","last_clean":"2024-09-18T01:07:20.952512+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:20.952512+0000","last_undegraded":"2024-09-18T01:07:20.952512+0000","last_fullsized":"2024-09-18T01:07:20.952512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867098+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:21.867098+0000","last_peered":"2024-09-18T01:07:21.867098+0000","last_clean":"2024-09-18T01:07:21.867098+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:21.867098+0000","last_undegraded":"2024-09-18T01:07:21.867098+0000","last_fullsized":"2024-09-18T01:07:21.867098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:25.013 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:07:25.013 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T01:07:25.013 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T01:07:25.014 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T01:07:25.324 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:25.324 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T01:07:25.337 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T01:07:25.337 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T01:07:25.358 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T01:07:25.358 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-18T01:07:25.358 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T01:07:25.358 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T01:07:25.655 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T01:07:25.656 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:07:25.656 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:25.929 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:25.930 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:25.941 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T01:07:24.723041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219036,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2842624,"data_stored":1737870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.030442+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:21.030442+0000","last_peered":"2024-09-18T01:07:21.030442+0000","last_clean":"2024-09-18T01:07:21.030442+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:21.030442+0000","last_undegraded":"2024-09-18T01:07:21.030442+0000","last_fullsized":"2024-09-18T01:07:21.030442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954875+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:20.954875+0000","last_peered":"2024-09-18T01:07:20.954875+0000","last_clean":"2024-09-18T01:07:20.954875+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:20.954875+0000","last_undegraded":"2024-09-18T01:07:20.954875+0000","last_fullsized":"2024-09-18T01:07:20.954875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867669+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:21.867669+0000","last_peered":"2024-09-18T01:07:21.867669+0000","last_clean":"2024-09-18T01:07:21.867669+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:21.867669+0000","last_undegraded":"2024-09-18T01:07:21.867669+0000","last_fullsized":"2024-09-18T01:07:21.867669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954917+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:20.954917+0000","last_peered":"2024-09-18T01:07:20.954917+0000","last_clean":"2024-09-18T01:07:20.954917+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:20.954917+0000","last_undegraded":"2024-09-18T01:07:20.954917+0000","last_fullsized":"2024-09-18T01:07:20.954917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.953958+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:20.953958+0000","last_peered":"2024-09-18T01:07:20.953958+0000","last_clean":"2024-09-18T01:07:20.953958+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:20.953958+0000","last_undegraded":"2024-09-18T01:07:20.953958+0000","last_fullsized":"2024-09-18T01:07:20.953958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.949080+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:20.949080+0000","last_peered":"2024-09-18T01:07:20.949080+0000","last_clean":"2024-09-18T01:07:20.949080+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:20.949080+0000","last_undegraded":"2024-09-18T01:07:20.949080+0000","last_fullsized":"2024-09-18T01:07:20.949080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867555+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:21.867555+0000","last_peered":"2024-09-18T01:07:21.867555+0000","last_clean":"2024-09-18T01:07:21.867555+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:21.867555+0000","last_undegraded":"2024-09-18T01:07:21.867555+0000","last_fullsized":"2024-09-18T01:07:21.867555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.952512+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:20.952512+0000","last_peered":"2024-09-18T01:07:20.952512+0000","last_clean":"2024-09-18T01:07:20.952512+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:20.952512+0000","last_undegraded":"2024-09-18T01:07:20.952512+0000","last_fullsized":"2024-09-18T01:07:20.952512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867098+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:21.867098+0000","last_peered":"2024-09-18T01:07:21.867098+0000","last_clean":"2024-09-18T01:07:21.867098+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:21.867098+0000","last_undegraded":"2024-09-18T01:07:21.867098+0000","last_fullsized":"2024-09-18T01:07:21.867098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:25.942 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:26.217 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:26.218 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:26.229 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T01:07:24.723041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219036,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2842624,"data_stored":1737870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.030442+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:21.030442+0000","last_peered":"2024-09-18T01:07:21.030442+0000","last_clean":"2024-09-18T01:07:21.030442+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:21.030442+0000","last_undegraded":"2024-09-18T01:07:21.030442+0000","last_fullsized":"2024-09-18T01:07:21.030442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954875+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:20.954875+0000","last_peered":"2024-09-18T01:07:20.954875+0000","last_clean":"2024-09-18T01:07:20.954875+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:20.954875+0000","last_undegraded":"2024-09-18T01:07:20.954875+0000","last_fullsized":"2024-09-18T01:07:20.954875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867669+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:21.867669+0000","last_peered":"2024-09-18T01:07:21.867669+0000","last_clean":"2024-09-18T01:07:21.867669+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:21.867669+0000","last_undegraded":"2024-09-18T01:07:21.867669+0000","last_fullsized":"2024-09-18T01:07:21.867669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.954917+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:20.954917+0000","last_peered":"2024-09-18T01:07:20.954917+0000","last_clean":"2024-09-18T01:07:20.954917+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:20.954917+0000","last_undegraded":"2024-09-18T01:07:20.954917+0000","last_fullsized":"2024-09-18T01:07:20.954917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.953958+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:20.953958+0000","last_peered":"2024-09-18T01:07:20.953958+0000","last_clean":"2024-09-18T01:07:20.953958+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:20.953958+0000","last_undegraded":"2024-09-18T01:07:20.953958+0000","last_fullsized":"2024-09-18T01:07:20.953958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.949080+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:20.949080+0000","last_peered":"2024-09-18T01:07:20.949080+0000","last_clean":"2024-09-18T01:07:20.949080+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:20.949080+0000","last_undegraded":"2024-09-18T01:07:20.949080+0000","last_fullsized":"2024-09-18T01:07:20.949080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867555+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:21.867555+0000","last_peered":"2024-09-18T01:07:21.867555+0000","last_clean":"2024-09-18T01:07:21.867555+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:21.867555+0000","last_undegraded":"2024-09-18T01:07:21.867555+0000","last_fullsized":"2024-09-18T01:07:21.867555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:20.952512+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:20.952512+0000","last_peered":"2024-09-18T01:07:20.952512+0000","last_clean":"2024-09-18T01:07:20.952512+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:20.952512+0000","last_undegraded":"2024-09-18T01:07:20.952512+0000","last_fullsized":"2024-09-18T01:07:20.952512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T01:07:21.867098+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:21.867098+0000","last_peered":"2024-09-18T01:07:21.867098+0000","last_clean":"2024-09-18T01:07:21.867098+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:21.867098+0000","last_undegraded":"2024-09-18T01:07:21.867098+0000","last_fullsized":"2024-09-18T01:07:21.867098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":65263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:26.230 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:07:26.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T01:07:26.506 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 22 2024-09-18T01:07:26.506 INFO:teuthology.orchestra.run.smithi120.stdout:fsid e017e0c5-d4e1-4f77-8127-47c02525e65d 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-18T01:06:59.746320+0000 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-18T01:07:24.528465+0000 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:crush_version 5 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:full_ratio 0.95 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:backfillfull_ratio 0.9 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:nearfull_ratio 0.85 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:require_min_compat_client luminous 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:min_compat_client jewel 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:require_osd_release squid 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:stretch_mode_enabled false 2024-09-18T01:07:26.507 INFO:teuthology.orchestra.run.smithi120.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:max_osd 8 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1379550312,v1:172.21.15.120:6825/1379550312] [v2:172.21.15.120:6826/1379550312,v1:172.21.15.120:6827/1379550312] exists,up 13ff2fc9-b863-45cb-b111-62d2ff06dff1 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/2892053774,v1:172.21.15.120:6817/2892053774] [v2:172.21.15.120:6818/2892053774,v1:172.21.15.120:6819/2892053774] exists,up 01867b0a-aeb8-4b92-897b-f8176abf95a9 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/2679997431,v1:172.21.15.120:6809/2679997431] [v2:172.21.15.120:6810/2679997431,v1:172.21.15.120:6811/2679997431] exists,up 788e3030-dfc0-45b5-9c99-763dc9283283 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/712549013,v1:172.21.15.120:6801/712549013] [v2:172.21.15.120:6802/712549013,v1:172.21.15.120:6803/712549013] exists,up 9ecd2117-0d0b-4629-9e54-3578a68fa0db 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/3636718449,v1:172.21.15.162:6818/3636718449] [v2:172.21.15.162:6820/3636718449,v1:172.21.15.162:6822/3636718449] exists,up 3af2d4ee-bce6-49df-a10e-f7c367d9fcb5 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6817/3011845137,v1:172.21.15.162:6819/3011845137] [v2:172.21.15.162:6821/3011845137,v1:172.21.15.162:6823/3011845137] exists,up fbac8f09-7655-4b5e-8bb5-74650aa3c2f1 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/4172964327,v1:172.21.15.162:6809/4172964327] [v2:172.21.15.162:6810/4172964327,v1:172.21.15.162:6811/4172964327] exists,up 1a98a211-cab7-414a-ad17-b06cede06bcf 2024-09-18T01:07:26.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/3795653175,v1:172.21.15.162:6801/3795653175] [v2:172.21.15.162:6802/3795653175,v1:172.21.15.162:6803/3795653175] exists,up 494f3743-c1f1-410f-b5f8-ce5927494495 2024-09-18T01:07:26.519 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1379550312,v1:172.21.15.120:6825/1379550312] [v2:172.21.15.120:6826/1379550312,v1:172.21.15.120:6827/1379550312] exists,up 13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'osd.1 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/2892053774,v1:172.21.15.120:6817/2892053774] [v2:172.21.15.120:6818/2892053774,v1:172.21.15.120:6819/2892053774] exists,up 01867b0a-aeb8-4b92-897b-f8176abf95a9', 'osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/2679997431,v1:172.21.15.120:6809/2679997431] [v2:172.21.15.120:6810/2679997431,v1:172.21.15.120:6811/2679997431] exists,up 788e3030-dfc0-45b5-9c99-763dc9283283', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/712549013,v1:172.21.15.120:6801/712549013] [v2:172.21.15.120:6802/712549013,v1:172.21.15.120:6803/712549013] exists,up 9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/3636718449,v1:172.21.15.162:6818/3636718449] [v2:172.21.15.162:6820/3636718449,v1:172.21.15.162:6822/3636718449] exists,up 3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6817/3011845137,v1:172.21.15.162:6819/3011845137] [v2:172.21.15.162:6821/3011845137,v1:172.21.15.162:6823/3011845137] exists,up fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/4172964327,v1:172.21.15.162:6809/4172964327] [v2:172.21.15.162:6810/4172964327,v1:172.21.15.162:6811/4172964327] exists,up 1a98a211-cab7-414a-ad17-b06cede06bcf', 'osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/3795653175,v1:172.21.15.162:6801/3795653175] [v2:172.21.15.162:6802/3795653175,v1:172.21.15.162:6803/3795653175] exists,up 494f3743-c1f1-410f-b5f8-ce5927494495'] 2024-09-18T01:07:26.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T01:07:26.617 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mon.a", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster": "ceph", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "admin_socket_mode": "", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_client_required": "cephx, none", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_cluster_required": "cephx", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_debug": "true", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_service_required": "cephx", 2024-09-18T01:07:26.618 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_supported": "cephx", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio": "true", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_reap_max": "16", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_async_discard": "false", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_block_size": "4096", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio": "false", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_enable_discard": "false", 2024-09-18T01:07:26.619 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_flock_retry": "3", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_inject_crash": "0", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring": "false", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring_hipri": "false", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-18T01:07:26.620 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_type": "", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_allocator": "hybrid", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_buffered_io": "true", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T01:07:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_replay_recovery": "false", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_sync_write": "false", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_allocator": "hybrid", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T01:07:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_multi": "true", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_create": "true", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_create": "false", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_path": "", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_size": "0", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_path": "", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T01:07:26.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_size": "107374182400", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_create": "false", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_path": "", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs": "true", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:07:26.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_autotune": "true", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size": "0", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_type": "2q", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_clone_cow": "true", 2024-09-18T01:07:26.625 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_mode": "none", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_freelist": "false", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T01:07:26.626 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T01:07:26.627 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T01:07:26.628 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fail_eio": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T01:07:26.629 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T01:07:26.630 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size": "0", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T01:07:26.631 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_mem": "512", 2024-09-18T01:07:26.632 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_tracing": "false", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T01:07:26.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T01:07:26.634 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_cluster_require_version": "2", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_require_signatures": "false", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_require_version": "2", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_service_require_signatures": "false", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_service_require_version": "2", 2024-09-18T01:07:26.635 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_sign_messages": "true", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "chdir": "", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_acl_type": "", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_asio_thread_count": "2", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_cache_mid": "0.750000", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_cache_size": "16384", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_caps_release_delay": "5", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_check_pool_perm": "true", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_force_sync_read": "false", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_getattr_caps": "false", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_inject_features": "", 2024-09-18T01:07:26.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_die_on_failed_remount": "false", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_dirsize_rbytes": "true", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_force_lazyio": "false", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_fs": "", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_inject_release_failure": "false", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_max_inline_size": "4096", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mds_namespace": "", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_metadata": "", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_gid": "-1", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_timeout": "300", 2024-09-18T01:07:26.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_uid": "-1", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mountpoint": "/", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_notify_timeout": "10", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc": "true", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_objects": "1000", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_size": "209715200", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_permissions": "true", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_quota": "true", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_quota_df": "true", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_max_bytes": "0", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_max_periods": "4", 2024-09-18T01:07:26.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_min": "131072", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_reconnect_stale": "false", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_shutdown_timeout": "30", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_snapdir": ".snap", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_tick_interval": "1", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_trace": "", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_use_faked_inos": "false", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_use_random_mds": "false", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog": "false", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog_port": "12201", 2024-09-18T01:07:26.639 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_monitors": "default=true", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog": "false", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog_level": "info", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "-", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_network": "", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_network_interface": "", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_isal": "false", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_level": "5", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zstd_level": "1", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T01:07:26.640 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_stat_interval": "0", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_seastar_cpu_cores": "", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_seastar_num_threads": "0", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location": "", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location_hook": "", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location_hook_timeout": "10", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "daemonize": "false", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "daos_pool": "tank", 2024-09-18T01:07:26.641 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_alienstore": "0/5", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asok": "1/5", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asok_assert_abort": "false", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_auth": "20/20", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bdev": "1/3", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bluefs": "1/5", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bluestore": "1/5", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_buffer": "0/1", 2024-09-18T01:07:26.642 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cephsqlite": "0/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_client": "0/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_compressor": "1/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_context": "0/1", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crimson_interrupt": "0/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crush": "1/1", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crypto": "1/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cyanstore": "0/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_dpdk": "1/5", 2024-09-18T01:07:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_eventtrace": "1/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_filer": "0/1", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_filestore": "1/3", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_finisher": "1/1", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_fuse": "1/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_javaclient": "1/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_journal": "1/3", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_journaler": "0/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_kstore": "1/5", 2024-09-18T01:07:26.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_lockdep": "0/1", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mclock": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_balancer": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_locker": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_log": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_migrator": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_quiesce": "3/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_memstore": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mgr": "2/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mgrc": "1/5", 2024-09-18T01:07:26.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mon": "20/20", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_monc": "0/10", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_ms": "1/1", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_none": "0/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objclass": "0/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objectcacher": "0/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objecter": "0/1", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_optracker": "0/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_osd": "1/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_paxos": "20/20", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_perfcounter": "1/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_prioritycache": "1/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rados": "0/5", 2024-09-18T01:07:26.646 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd": "0/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_replay": "0/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_refs": "0/0", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_reserver": "1/1", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_access": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_dbstore": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_flight": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_sync": "1/5", 2024-09-18T01:07:26.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rocksdb": "4/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_backref": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_cache": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_device": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_epm": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_journal": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_lba": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_odata": "0/5", 2024-09-18T01:07:26.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_omap": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_onode": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_t": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_tm": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_striper": "0/1", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_test": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_throttle": "1/1", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_timer": "0/1", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_tp": "0/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_trace": "1/5", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "device_failure_prediction_mode": "none", 2024-09-18T01:07:26.649 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_graylog": "false", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_journald": "false", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_stderr": "true", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_syslog": "false", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event_tracing": "false", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_http_port": "9926", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_prio_limit": "5", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_sort_metrics": "true", 2024-09-18T01:07:26.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_stats_period": "5", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "fake_statfs_for_testing": "0", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "fatal_signal_handlers": "true", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filer_max_purge_ops": "10", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filer_max_truncate_ops": "128", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_blackhole": "false", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_btrfs_snap": "true", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_caller_concurrency": "10", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T01:07:26.651 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_omap_check": "false", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_verify_split": "false", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_dump_file": "", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fadvise": "true", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fail_eio": "true", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fd_cache_size": "128", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fiemap": "false", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T01:07:26.652 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_inject_stall": "0", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_parallel": "false", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_trailing": "false", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_writeahead": "false", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_kill_at": "0", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T01:07:26.653 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_merge_threshold": "-10", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_odsync_write": "false", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_backend_path": "", 2024-09-18T01:07:26.654 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_threads": "2", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_punch_hole": "false", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T01:07:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_ops": "50", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_seek_data_hole": "false", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_sloppy_crc": "false", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_splice": "false", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_split_multiple": "2", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_split_rand_factor": "20", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_update_to": "1000", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T01:07:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T01:07:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_xfs_extsize": "false", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_zfs_snap": "false", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fio_dir": "/tmp/fio", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_allow_other": "true", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_big_writes": "true", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_debug": "false", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_default_permissions": "false", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_disable_pagecache": "false", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_max_write": "0", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_multithreaded": "true", 2024-09-18T01:07:26.658 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_require_active_mds": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_set_user_groups": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_move": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_read": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_write": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "gss_target_name": "ceph", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_file": "", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_inject_failure": "0", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_interval": "5", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "host": "smithi120", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T01:07:26.659 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T01:07:26.660 INFO:teuthology.orchestra.run.smithi120.stdout: "inject_early_sigterm": "false", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "jaeger_agent_port": "6799", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "jaeger_tracing_enable": "false", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_aio": "true", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_align_min_size": "65536", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_block_align": "true", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_block_size": "4096", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_dio": "true", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_discard": "false", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_force_aio": "false", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_ignore_corruption": "false", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T01:07:26.661 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_max_write_entries": "100", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_replay_from": "0", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_write_header_frequency": "0", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_zero_on_create": "false", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_prefetch_periods": "10", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_prezero_periods": "5", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_write_head_interval": "15", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "key": "", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "keyfile": "", 2024-09-18T01:07:26.662 INFO:teuthology.orchestra.run.smithi120.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_backend": "rocksdb", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_max_bytes": "67108864", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_max_ops": "512", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_onode_map_size": "1024", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_sync_transaction": "false", 2024-09-18T01:07:26.663 INFO:teuthology.orchestra.run.smithi120.stdout: "librados_thread_count": "2", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "lockdep": "false", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "lockdep_force_backtrace": "false", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_coarse_timestamps": "true", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_flush_on_exit": "false", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_graylog_port": "12201", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_max_new": "1000", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_max_recent": "10000", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_stderr_prefix": "", 2024-09-18T01:07:26.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": "true", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_graylog": "false", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_journald": "false", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_stderr": "false", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_syslog": "false", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_action_on_write_error": "1", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_alternate_name_max": "8192", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_asio_thread_count": "2", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_export_pin": "true", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T01:07:26.665 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_interval": "10", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_max": "-1", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_max_until": "-1", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_merge_size": "50", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T01:07:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_mode": "0", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_overload_epochs": "2", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_bits": "3", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_size": "10000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T01:07:26.667 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_mid": "0.700000", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_delay": "0", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T01:07:26.668 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_interval": "1", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_auth_pins": "false", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_frag": "false", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_scatterstat": "false", 2024-09-18T01:07:26.669 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_subtrees": "false", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_default_dir_hash": "2", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_defer_session_stale": "true", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_max_entries": "0", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_prefetch": "true", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T01:07:26.670 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_early_reply": "true", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_enable_op_tracker": "true", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_enforce_unique_name": "true", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T01:07:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_health_dummy": "false", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T01:07:26.672 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_join_fs": "", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_journal_format": "1", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_create_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_export_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_import_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_link_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T01:07:26.673 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_openc_at": "0", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_rename_at": "0", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_shutdown_at": "0", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_event_large_threshold": "524288", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_max_events": "-1", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_max_segments": "128", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_pause": "false", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_segment_size": "0", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-18T01:07:26.674 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_threshold": "128", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_completed_requests": "100000", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_export_size": "20971520", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_file_recover": "32", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_files": "64", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_ops": "8192", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T01:07:26.675 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_metrics_update_interval": "2", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_min_caps_per_client": "100", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_numa_node": "-1", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_duration": "600", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_size": "20", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_log_threshold": "5", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_ping_grace": "15", 2024-09-18T01:07:26.676 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_ping_interval": "5", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_caps": "30000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_replay_interval": "1.000000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_root_ino_gid": "0", 2024-09-18T01:07:26.677 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_root_ino_uid": "0", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_shutdown_check": "0", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_skip_ino": "0", 2024-09-18T01:07:26.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_min_uid": "0", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_rstat": "false", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_symlink_recovery": "true", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_thrash_exports": "0", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_thrash_fragments": "0", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_tick_interval": "5.000000", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_valgrind_exit": "false", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_verify_backtrace": "1", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_verify_scatter": "false", 2024-09-18T01:07:26.679 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_wipe_sessions": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool_debug": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_page_set": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_page_size": "65536", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_bytes": "134217728", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_messages": "512", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T01:07:26.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_disabled_modules": "", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_enable_op_tracker": "true", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mds_messages": "128", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mon_messages": "128", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_duration": "600", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_size": "20", 2024-09-18T01:07:26.681 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_log_threshold": "5", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_osd_messages": "8192", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_pool": "true", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_standby_modules": "true", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_stats_period": "5", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_stats_threshold": "5", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_tick_period": "2", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T01:07:26.682 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_allow_pool_delete": "true", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_bytes": "104857600", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T01:07:26.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_target_rank": "-1", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_file_level": "debug", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T01:07:26.684 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_start": "false", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_trim": "true", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T01:07:26.685 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cpu_threads": "4", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_avail_crit": "5", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_avail_warn": "5", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_json": "false", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_extra_checks": "false", 2024-09-18T01:07:26.686 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_reef": "false", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_squid": "false", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_down_uptime_grace": "60", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_election_default_strategy": "1", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_election_timeout": "5.000000", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T01:07:26.687 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_enable_op_tracker": "true", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_fake_pool_delete": "false", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_force_quorum_join": "false", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_log_update_period": "5", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_max_detail": "50", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog": "true", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T01:07:26.688 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_host": "172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790],172.21.15.162", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_host_override": "", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_initial_members": "", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease": "5.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_full_interval": "50", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_max": "10000", 2024-09-18T01:07:26.689 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_max_summary": "50", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_log_epochs": "500", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_osd": "10000", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_memory_autotune": "true", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_memory_target": "2147483648", 2024-09-18T01:07:26.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_digest_period": "5", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_complaint_time": "30", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_duration": "600", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_size": "20", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T01:07:26.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_log_threshold": "5", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_cache_size": "500", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T01:07:26.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T01:07:26.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_report_timeout": "900", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T01:07:26.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_max_osds": "4", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T01:07:26.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_interval": "86400", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_max_keys": "100", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_timeout": "300", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_session_timeout": "300", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_smart_report_timeout": "5", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T01:07:26.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_pool_size": "4", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_debug": "false", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_tick_interval": "5", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T01:07:26.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_use_min_delay_socket": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_older_version": "true", 2024-09-18T01:07:26.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "monmap": "", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-18T01:07:26.699 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_tracing_enabled": "false", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_op_threads": "3", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_cm": "false", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_device_name": "", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T01:07:26.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_sl": "3", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_type": "ib", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_reap_threshold": "5", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_before_connect": "false", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_ipv4": "true", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_ipv6": "false", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_msgr1": "true", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_msgr2": "true", 2024-09-18T01:07:26.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_port_max": "7568", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_port_min": "6800", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_retry_count": "3", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_retry_delay": "5", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_client": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mds": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mgr": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mon": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_osd": "false", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_client_mode": "crc secure", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_cluster_type": "", 2024-09-18T01:07:26.702 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_compress_secure": "false", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_crc_data": "true", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_crc_header": "true", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_bug": "true", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_old_message": "true", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T01:07:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hugepages": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_lro": "true", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_pmd": "", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_port_id": "0", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T01:07:26.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dump_on_send": "false", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_type": "", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_network_congestion": "0", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_max_accept_failures": "4", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_max_backoff": "15.000000", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T01:07:26.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compress_mode": "none", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_pq_min_cost": "65536", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_public_type": "", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_service_mode": "crc secure", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_nodelay": "true", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T01:07:26.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_type": "async+posix", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "no_config_file": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inflight_ops": "1024", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_timeout": "10.000000", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objectstore_blackhole": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "openssl_engine_opts": "", 2024-09-18T01:07:26.707 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_max_ops": "4", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_slop": "0.020000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_asio_thread_count": "2", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T01:07:26.708 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_scan_max": "512", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_scan_min": "64", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_peering": "false", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_beacon_report_interval": "300", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_duration": "30", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_blkin_trace_all": "false", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T01:07:26.709 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_update_on_start": "true", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_message_cap": "256", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_op_priority": "63", 2024-09-18T01:07:26.710 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_watch_timeout": "30", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_command_max_records": "256", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_compact_on_start": "false", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_update_on_start": "true", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T01:07:26.711 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_op_order": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T01:07:26.712 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_shutdown": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_trim_objects": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T01:07:26.713 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_default_notify_timeout": "30", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_enable_op_tracker": "true", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T01:07:26.714 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_info": "true", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown": "true", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_function_tracing": "false", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_grace": "20", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_interval": "6", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T01:07:26.715 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_stale": "600", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal_size": "5120", 2024-09-18T01:07:26.716 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_kill_backfill_at": "0", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_cache_size": "50", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_dedup": "true", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_message_max": "40", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_attr_name_len": "100", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_attr_size": "0", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_backfills": "1", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_markdown_count": "5", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_markdown_period": "600", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_name_len": "2048", 2024-09-18T01:07:26.717 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_size": "134217728", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pgls": "1024", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_push_cost": "8388608", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_push_objects": "10", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_scrubs": "3", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T01:07:26.718 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_write_size": "90", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_profile": "balanced", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T01:07:26.719 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_base": "805306368", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T01:07:26.720 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target": "4294967296", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target_autotune": "false", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_min_recovery_priority": "0", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_report_interval": "5", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_num_cache_shards": "32", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T01:07:26.721 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_node": "-1", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objecter_finishers": "1", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore": "bluestore", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore_fuse": "false", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore_tracing": "false", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_duration": "600", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_size": "20", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_log_threshold": "5", 2024-09-18T01:07:26.722 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards": "0", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_thread_timeout": "15", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_open_classes_on_start": "true", 2024-09-18T01:07:26.723 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_os_flags": "0", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_peering_op_priority": "255", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_delete_priority": "5", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T01:07:26.724 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_crimson": "false", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T01:07:26.725 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flags": "0", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_min_size": "0", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_read_ratio": "70", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_size": "2", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_type": "replicated", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T01:07:26.726 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_cost": "20971520", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active": "0", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_op_priority": "3", 2024-09-18T01:07:26.727 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_priority": "5", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_repair_during_recovery": "false", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T01:07:26.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_cost": "52428800", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_end_hour": "0", 2024-09-18T01:07:26.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_priority": "5", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-18T01:07:26.730 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_skip_data_digest": "false", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_smart_report_timeout": "5", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_priority": "5", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-18T01:07:26.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_target_transaction_size": "30", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T01:07:26.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tracing": "false", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_use_stale_snap": "false", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_kill_at": "0", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_max_join_drift": "10", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_min": "500", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_min_wait": "0.050000", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_max": "500", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_min": "250", 2024-09-18T01:07:26.733 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_stash_full_interval": "25", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_trim_max": "500", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_trim_min": "250", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "perf": "true", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "-", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrv": "", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "public_bind_addr": "-", 2024-09-18T01:07:26.734 INFO:teuthology.orchestra.run.smithi120.stdout: "public_network": "", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "public_network_interface": "", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_busy_polling": "false", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_enabled": "false", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_session_max_number": "256", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_mon_op_timeout": "0", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_osd_op_timeout": "0", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_tracing": "false", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_atime_update_interval": "60", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T01:07:26.735 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache": "true", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_policy": "writearound", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_size": "33554432", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T01:07:26.736 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_compression_hint": "none", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_clone_format": "auto", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_data_pool": "", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_features": "61", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_format": "2", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_map_options": "", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_order": "22", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_pool": "rbd", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T01:07:26.737 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_stripe_count": "0", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_io_scheduler": "simple", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T01:07:26.738 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_order": "24", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_pool": "", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_splay_width": "4", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T01:07:26.739 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T01:07:26.740 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T01:07:26.741 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_op_threads": "1", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T01:07:26.742 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_plugins": "", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T01:07:26.743 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T01:07:26.744 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_tracing": "false", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_validate_names": "true", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_validate_pool": "true", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "restapi_base_url": "", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "restapi_log_level": "", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_admin_entry": "admin", 2024-09-18T01:07:26.745 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_backend_store": "rados", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_barbican_url": "", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_beast_enable_async": "true", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_counters_cache": "false", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T01:07:26.746 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_enabled": "true", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_lru_size": "25000", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_config_store": "rados", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_content_length_compat": "false", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cross_domain_policy": "", 2024-09-18T01:07:26.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T01:07:26.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T01:07:26.749 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T01:07:26.750 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d4n_port": "6379", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T01:07:26.751 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_window": "30", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T01:07:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T01:07:26.753 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dns_name": "", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dns_s3website_name": "", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_ops_log": "false", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T01:07:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_static_website": "false", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_usage_log": "false", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_expose_bucket": "false", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_extended_http_attrs": "", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_filter": "none", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_objs": "32", 2024-09-18T01:07:26.755 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_init_timeout": "300", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T01:07:26.756 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-18T01:07:26.757 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T01:07:26.757 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T01:07:26.757 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-18T01:07:26.761 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_password": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_project": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_token": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_user": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_api_version": "2", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T01:07:26.762 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_url": "", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T01:07:26.763 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_objs": "32", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_worker": "3", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_uri": "", 2024-09-18T01:07:26.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_http_headers": "", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attr_size": "0", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T01:07:26.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_listing_results": "1000", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_notify_retries": "10", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T01:07:26.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T01:07:26.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_num_control_oids": "8", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_numa_node": "-1", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T01:07:26.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_tracing": "false", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_token": "", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_url": "", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_rados": "false", 2024-09-18T01:07:26.769 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_lanes": "3", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-18T01:07:26.770 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_partitions": "3", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_print_continue": "true", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_tracing": "false", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm": "", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm_id": "", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T01:07:26.771 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_region": "", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_request_uri": "", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_resolve_cname": "false", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T01:07:26.772 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_run_sync_thread": "true", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_script_uri": "", 2024-09-18T01:07:26.773 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_service_provider_name": "", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_client_id": "", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_client_secret": "", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_entry": "sts", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_key": "sts", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_auth_url": "", 2024-09-18T01:07:26.774 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_custom_header": "", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_need_stats": "true", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_tenant_name": "", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_url": "", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_lease_period": "120", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T01:07:26.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_thread_pool_size": "512", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-18T01:07:26.776 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_comment": "", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_createby": "", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_encoding": "", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_flag": "false", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_origin": "", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_tracker": "", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_max_shards": "32", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T01:07:26.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_use_opa_authz": "false", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_counters_cache": "false", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_policies_max_num": "100", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T01:07:26.778 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_unique_email": "true", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_verify_ssl": "true", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone": "", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone_id": "", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup": "", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup_id": "", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgwlc_auto_session_clear": "true", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-18T01:07:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_block_size": "4096", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T01:07:26.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_partition_filters": "false", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_perf": "false", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "run_dir": "/var/run/ceph", 2024-09-18T01:07:26.781 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_block_create": "true", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_device_size": "53687091200", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_full_integrity_check": "false", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T01:07:26.782 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_main_device_type": "SSD", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_segment_size": "67108864", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "set_keepcaps": "false", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "setgroup": "", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "setuser": "", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "setuser_match_path": "", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T01:07:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: "thp": "false", 2024-09-18T01:07:26.784 INFO:teuthology.orchestra.run.smithi120.stdout: "threadpool_default_timeout": "60", 2024-09-18T01:07:26.784 INFO:teuthology.orchestra.run.smithi120.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T01:07:26.784 INFO:teuthology.orchestra.run.smithi120.stdout: "throttler_perf_counter": "true" 2024-09-18T01:07:26.784 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:26.786 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T01:07:26.897 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a: {} 2024-09-18T01:07:26.897 INFO:teuthology.orchestra.run.smithi120.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:07:26.906 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b: {} 2024-09-18T01:07:26.907 INFO:teuthology.orchestra.run.smithi120.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:07:26.915 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c: {} 2024-09-18T01:07:26.915 INFO:teuthology.orchestra.run.smithi120.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:07:26.926 DEBUG:teuthology.orchestra.run.smithi120:> type ceph-objectstore-tool 2024-09-18T01:07:26.940 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T01:07:26.940 DEBUG:teuthology.orchestra.run.smithi162:> type ceph-objectstore-tool 2024-09-18T01:07:26.958 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T01:07:26.959 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-18T01:07:26.970 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-18T01:07:26.970 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T01:07:26.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:07:26.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:07:26.971 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:07:26.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T01:07:26.971 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T01:07:26.971 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T01:07:26.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:26.972 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T01:07:26.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:26.972 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T01:07:26.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:07:26.973 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 -g pool_client.0 image_client.0 2024-09-18T01:07:27.000 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:26.999+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:27.019 INFO:teuthology.orchestra.run.smithi120.stdout:mapped writes DISABLED 2024-09-18T01:07:27.019 INFO:teuthology.orchestra.run.smithi120.stdout:mapped reads DISABLED 2024-09-18T01:07:27.019 INFO:teuthology.orchestra.run.smithi120.stdout:Seed set to 9637 2024-09-18T01:07:27.073 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:27.072+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:27.094 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:27.094 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:27.094 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:27.094 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:27.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:27.118 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:27.119 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:27.145 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:27.145 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:27.161 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:27.174 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:27.173+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:27.179 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:27.180 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:27.197 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:27.215 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:27.227 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.227 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.232 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:27.236 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:27.249 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:27.303 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:27.302+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:27.307 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 18 (0%) 2024-09-18T01:07:27.307 INFO:teuthology.orchestra.run.smithi120.stdout:avg 2.25 2024-09-18T01:07:27.307 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 1.29904 -> 1.29904 (expected baseline 1.40312) 2024-09-18T01:07:27.307 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.4 with 0 -> 0 pgs (0 -> 0 * mean) 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.1 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 1 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0003 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0003 2024-09-18T01:07:27.308 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:07:27.320 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:07:27.320 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-18T01:07:27.354 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.354 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:27.405 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:27.404+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:27.477 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:27.476+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:27.510 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.510 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:27.580 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:27.577+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:27.623 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.624 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 18 (0%) 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout:avg 2.25 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 1.29904 -> 1.29904 (expected baseline 1.40312) 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.4 with 0 -> 0 pgs (0 -> 0 * mean) 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.1 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-09-18T01:07:27.639 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 0.05 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0003 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0003 2024-09-18T01:07:27.640 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:07:27.653 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:07:27.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-18T01:07:27.679 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:27.677+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:27.745 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.745 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.755 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:27.779 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:27.778+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:27.870 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.870 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.879 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:27.880 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:27.879+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:27.966 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:07:27.982 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:27.981+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:27.989 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:27.989 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:27.998 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:28.006 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:07:28.006 INFO:teuthology.orchestra.run.smithi120.stdout:avg 10.25 2024-09-18T01:07:28.006 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-18T01:07:28.006 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-18T01:07:28.006 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 1 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0003 2024-09-18T01:07:28.007 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0003 2024-09-18T01:07:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:07:28.018 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:07:28.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T01:07:28.103 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:28.102+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:28.112 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.112 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.121 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:28.184 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:28.183+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:28.267 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.267 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:28.284 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:28.283+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout:avg 10.25 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T01:07:28.344 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 3 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0003 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0003 2024-09-18T01:07:28.345 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:07:28.357 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T01:07:28.358 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-18T01:07:28.380 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.380 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.385 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:28.384+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:28.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:28.486 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:28.485+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:28.538 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.538 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.547 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:28.589 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:28.588+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:28.645 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:07:28.645 INFO:teuthology.orchestra.run.smithi120.stdout:avg 10.25 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:oload 1000 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 1 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0003 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0029 2024-09-18T01:07:28.646 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:07:28.664 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.664 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.673 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:28.718 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:28.717+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:28.778 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.779 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.787 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:28.789 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:28.788+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:28.889 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:28.888+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:28.930 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:28.930 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:28.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:29.023 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:29.022+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:29.083 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:29.083 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:29.091 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:29.092 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:29.090+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:29.192 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:29.191+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:29.235 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:29.235 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:29.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:29.262 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:29.293 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:29.292+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:29.349 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:29.349 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:29.357 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:29.382 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:29.382 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:29.393 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:29.392+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:29.398 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:29.399 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:29.417 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:29.430 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:29.430 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:29.446 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:29.446 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:29.463 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:29.463 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:29.479 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:29.479 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:29.494 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:29.493+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:29.495 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:29.496 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:29.502 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:29.502 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:29.502 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:29.502 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:29.511 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:29.595 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:29.594+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:29.654 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:29.655 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:29.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:29.707 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:29.706+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:29.767 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:29.768 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:29.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:29.768 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [] 2024-09-18T01:07:29.768 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:29.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:29.797 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:29.797+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:29.898 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:29.897+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:29.919 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:29.920 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:29.920 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:29.920 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:29.929 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:29.989 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:07:30.034 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:30.033+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:30.073 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:30.073 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:30.081 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:30.100 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:30.099+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.287766019999999, 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:07:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:07:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:12.967298+0000", 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.218276485000001, 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067696359999999999, 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967298+0000", 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967298+0000", 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967299+0000", 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967301+0000", 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0669999999999999e-06 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967305+0000", 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5839999999999996e-06 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.967308+0000", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9289999999999998e-06 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.971263+0000", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039552709999999998 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.971341+0000", 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.737e-05 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.971469+0000", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012860700000000001 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.972384+0000", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00091472700000000005 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.974022+0000", 2024-09-18T01:07:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016377119999999999 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.974048+0000", 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6044999999999999e-05 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.974068+0000", 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9993e-05 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:12.988601+0000", 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.196973989, 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006834835, 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 85 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988601+0000", 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988601+0000", 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988601+0000", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1999999999999999e-07 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988605+0000", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8210000000000003e-06 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988613+0000", 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4800000000000004e-06 2024-09-18T01:07:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988618+0000", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9640000000000002e-06 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990180+0000", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015620359999999999 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990242+0000", 2024-09-18T01:07:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2830999999999995e-05 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990316+0000", 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3481999999999999e-05 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.994676+0000", 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043598589999999998 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995341+0000", 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066487299999999996 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995396+0000", 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5163e-05 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995436+0000", 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9706000000000002e-05 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:12.988627+0000", 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.1969475, 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00689837, 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 86 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988627+0000", 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988627+0000", 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988628+0000", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.15e-07 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988630+0000", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4559999999999999e-06 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988634+0000", 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999997e-06 2024-09-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988637+0000", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9440000000000001e-06 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990723+0000", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020855050000000001 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990770+0000", 2024-09-18T01:07:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7859999999999999e-05 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.990844+0000", 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3069000000000001e-05 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.994678+0000", 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038348549999999999 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995476+0000", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00079734000000000001 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995505+0000", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9009e-05 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995526+0000", 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0817000000000001e-05 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:12.988641+0000", 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.196933380000001, 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070082699999999996, 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 87 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988641+0000", 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988641+0000", 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988642+0000", 2024-09-18T01:07:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.96e-07 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988670+0000", 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7963999999999999e-05 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988674+0000", 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3470000000000001e-06 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988678+0000", 2024-09-18T01:07:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0210000000000001e-06 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.991135+0000", 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024566219999999999 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.991227+0000", 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2841000000000001e-05 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.991347+0000", 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000119355 2024-09-18T01:07:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995000+0000", 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036534480000000001 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995564+0000", 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00056362800000000005 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995613+0000", 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8869000000000003e-05 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995650+0000", 2024-09-18T01:07:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6878999999999997e-05 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:12.988683+0000", 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.196891344000001, 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.007045333, 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 88 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988683+0000", 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988683+0000", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988684+0000", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988686+0000", 2024-09-18T01:07:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1859999999999999e-06 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988689+0000", 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4809999999999998e-06 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.988692+0000", 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5220000000000002e-06 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.992108+0000", 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034162749999999999 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.992158+0000", 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9696e-05 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.992235+0000", 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7058000000000007e-05 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995016+0000", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027808109999999998 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995681+0000", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066483000000000004 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995707+0000", 2024-09-18T01:07:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6750999999999998e-05 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.995729+0000", 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1369000000000002e-05 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112609+0000", 2024-09-18T01:07:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072966202, 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069230589999999996, 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 111 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112609+0000", 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112609+0000", 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112609+0000", 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.82e-07 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112613+0000", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9750000000000001e-06 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112620+0000", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112625+0000", 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:07:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.115316+0000", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026903080000000002 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.115364+0000", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8341999999999999e-05 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.115472+0000", 2024-09-18T01:07:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107928 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117518+0000", 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020461659999999999 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119464+0000", 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00194567 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119495+0000", 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1786999999999999e-05 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119532+0000", 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6130000000000001e-05 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112638+0000", 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072936981000002, 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069648599999999998, 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 112 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112638+0000", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112638+0000", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112638+0000", 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4099999999999999e-07 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112676+0000", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7392999999999999e-05 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112685+0000", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1540000000000008e-06 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112690+0000", 2024-09-18T01:07:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.115817+0000", 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003126789 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.115895+0000", 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8448999999999994e-05 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.116027+0000", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131438 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117553+0000", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015265120000000001 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119540+0000", 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019865030000000001 2024-09-18T01:07:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119569+0000", 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9028999999999999e-05 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119603+0000", 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3781000000000001e-05 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112700+0000", 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072874792, 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069575699999999997, 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 113 2024-09-18T01:07:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112700+0000", 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112700+0000", 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112701+0000", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9800000000000003e-07 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112705+0000", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6999999999999999e-06 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112712+0000", 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9979999999999997e-06 2024-09-18T01:07:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112718+0000", 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8710000000000002e-06 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.116508+0000", 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037895369999999999 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.116558+0000", 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9908999999999999e-05 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.116662+0000", 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104603 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117586+0000", 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092378800000000004 2024-09-18T01:07:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119608+0000", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002022336 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119638+0000", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9907e-05 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119657+0000", 2024-09-18T01:07:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9323000000000001e-05 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112731+0000", 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072843523, 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069824049999999997, 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 114 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112731+0000", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112731+0000", 2024-09-18T01:07:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112732+0000", 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112734+0000", 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.886e-06 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112745+0000", 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0519e-05 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112750+0000", 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1390000000000003e-06 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117611+0000", 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048607060000000002 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117658+0000", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999998e-05 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117730+0000", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2545000000000004e-05 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119426+0000", 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001696066 2024-09-18T01:07:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119665+0000", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023884700000000001 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119692+0000", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6568999999999999e-05 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119714+0000", 2024-09-18T01:07:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2008e-05 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112800+0000", 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072774427999999, 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069649960000000002, 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 115 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112800+0000", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112800+0000", 2024-09-18T01:07:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112801+0000", 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4899999999999995e-07 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112841+0000", 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9629999999999998e-05 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112855+0000", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4664000000000001e-05 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112859+0000", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9489999999999998e-06 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.117955+0000", 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050960980000000003 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118004+0000", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8977000000000002e-05 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118078+0000", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3514000000000001e-05 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119457+0000", 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013790370000000001 2024-09-18T01:07:30.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119718+0000", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026112800000000001 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119748+0000", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9655999999999999e-05 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119765+0000", 2024-09-18T01:07:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7694e-05 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112877+0000", 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.07269784, 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006999323, 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 116 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112877+0000", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112877+0000", 2024-09-18T01:07:30.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112878+0000", 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7199999999999998e-07 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112882+0000", 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1320000000000004e-06 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112891+0000", 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9570000000000008e-06 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112896+0000", 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118483+0000", 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055865639999999996 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118566+0000", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3629999999999997e-05 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118647+0000", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1210000000000003e-05 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119797+0000", 2024-09-18T01:07:30.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011498280000000001 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119831+0000", 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4230999999999998e-05 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119856+0000", 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4896e-05 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119876+0000", 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9811999999999999e-05 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.112908+0000", 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.072666981000001, 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093817759999999997, 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.266 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 117 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112908+0000", 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112908+0000", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112908+0000", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112911+0000", 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-18T01:07:30.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112916+0000", 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9320000000000004e-06 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.112919+0000", 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.954e-06 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.118932+0000", 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060132349999999996 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119010+0000", 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8328e-05 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119134+0000", 2024-09-18T01:07:30.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012409199999999999 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.119825+0000", 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069086500000000003 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.122241+0000", 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002416011 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.122270+0000", 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8697e-05 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.122290+0000", 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9513000000000001e-05 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.506970+0000", 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.678604222000001, 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071652699999999996, 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 137 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.273 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.506970+0000", 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.506970+0000", 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.506971+0000", 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507024+0000", 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2868000000000003e-05 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507031+0000", 2024-09-18T01:07:30.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8839999999999997e-06 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507036+0000", 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.04e-06 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.510868+0000", 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003832355 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.510956+0000", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8054999999999996e-05 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.511084+0000", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000128116 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512575+0000", 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491322 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514064+0000", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014883030000000001 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514092+0000", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8201e-05 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514136+0000", 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3782000000000002e-05 2024-09-18T01:07:30.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507054+0000", 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.678520250999998, 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0073938220000000004, 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 138 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507054+0000", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507054+0000", 2024-09-18T01:07:30.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507055+0000", 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507059+0000", 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8890000000000004e-06 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507069+0000", 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0552e-05 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507074+0000", 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9219999999999996e-06 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.511640+0000", 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045653500000000001 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.511690+0000", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0491000000000002e-05 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.511766+0000", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5866999999999997e-05 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512609+0000", 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084260400000000005 2024-09-18T01:07:30.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514385+0000", 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017758990000000001 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514412+0000", 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7273000000000001e-05 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514448+0000", 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6507000000000001e-05 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507084+0000", 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.67849116, 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081797150000000006, 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 139 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507084+0000", 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507084+0000", 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507084+0000", 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507087+0000", 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507091+0000", 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1280000000000001e-06 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507094+0000", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.971e-06 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512634+0000", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055397190000000002 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512681+0000", 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7104000000000002e-05 2024-09-18T01:07:30.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512752+0000", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742000000000005e-05 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514454+0000", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017011280000000001 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.515206+0000", 2024-09-18T01:07:30.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00075269699999999998 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.515239+0000", 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2814999999999997e-05 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.515263+0000", 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4172000000000002e-05 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.678476114999999, 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010713336, 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 140 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507131+0000", 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2549000000000003e-05 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507136+0000", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5499999999999996e-06 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507139+0000", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512990+0000", 2024-09-18T01:07:30.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058514930000000001 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513036+0000", 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5995000000000003e-05 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513109+0000", 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2559999999999996e-05 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514456+0000", 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001347144 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517765+0000", 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00330846 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517791+0000", 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6574999999999999e-05 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517812+0000", 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0775999999999999e-05 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.678430072000001, 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010897142, 2024-09-18T01:07:30.298 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 141 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507148+0000", 2024-09-18T01:07:30.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507179+0000", 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1504000000000002e-05 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507185+0000", 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7089999999999999e-06 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513570+0000", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0063849830000000003 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513613+0000", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3689000000000001e-05 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513682+0000", 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8750000000000004e-05 2024-09-18T01:07:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514460+0000", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077749700000000004 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517975+0000", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035148150000000001 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518005+0000", 2024-09-18T01:07:30.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9986000000000001e-05 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518042+0000", 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:07:30.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.984479+0000", 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.201095385999999, 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067571879999999999, 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 158 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984479+0000", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984479+0000", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984480+0000", 2024-09-18T01:07:30.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984585+0000", 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105473 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984593+0000", 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984603+0000", 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.008e-05 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.986960+0000", 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002356707 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.987041+0000", 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1142000000000003e-05 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.987172+0000", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013096000000000001 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.988263+0000", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001091432 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991187+0000", 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029240849999999999 2024-09-18T01:07:30.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991214+0000", 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7007000000000001e-05 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991237+0000", 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2082e-05 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.984613+0000", 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 16.200961548999999, 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067526390000000004, 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 159 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984613+0000", 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984613+0000", 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:30.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984614+0000", 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2699999999999999e-07 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984617+0000", 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.658e-06 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984624+0000", 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6209999999999998e-06 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.984631+0000", 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5379999999999999e-06 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.987696+0000", 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030656709999999998 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.987780+0000", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3629999999999997e-05 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.987911+0000", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131316 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.988555+0000", 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064405300000000001 2024-09-18T01:07:30.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991276+0000", 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027210450000000001 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991329+0000", 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3189000000000002e-05 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:30.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.991366+0000", 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6390999999999998e-05 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:30.317 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:30.319 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:30.319 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:30.318+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:30.420 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:30.419+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:30.421 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:30.421 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:30.421 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:30.421 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:30.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:30.520 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:30.519+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:30.532 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:30.532 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:30.532 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:30.533 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:30.533 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:30.541 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:30.625 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:30.624+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:30.683 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.692553262000001, 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005693783, 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989266+0000", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989270+0000", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7190000000000001e-06 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989281+0000", 2024-09-18T01:07:30.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.063e-05 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989286+0000", 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989323+0000", 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7370000000000003e-05 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989351+0000", 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7874000000000001e-05 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994914+0000", 2024-09-18T01:07:30.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055624790000000004 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994959+0000", 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5194999999999997e-05 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.692064023, 2024-09-18T01:07:30.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052255940000000001, 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:30.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989761+0000", 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8539999999999999e-06 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989821+0000", 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0129e-05 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989831+0000", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4679999999999998e-06 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989835+0000", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989865+0000", 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0494999999999999e-05 2024-09-18T01:07:30.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989877+0000", 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.201e-05 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994961+0000", 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005083873 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994980+0000", 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8675999999999999e-05 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:85 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.691428395999999, 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046080890000000001, 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990394+0000", 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6389999999999998e-06 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990401+0000", 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6320000000000002e-06 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990404+0000", 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990436+0000", 2024-09-18T01:07:30.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1869999999999998e-05 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990449+0000", 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3097e-05 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994982+0000", 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004532683 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994998+0000", 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6056e-05 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'51)", 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.690900867, 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040959600000000001, 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990918+0000", 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990927+0000", 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5959999999999993e-06 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990934+0000", 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3430000000000001e-06 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990937+0000", 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6789999999999998e-06 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990966+0000", 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8986e-05 2024-09-18T01:07:30.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990979+0000", 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2351000000000001e-05 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995000+0000", 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040208450000000003 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995013+0000", 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3773e-05 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'51)", 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.690301856000001, 2024-09-18T01:07:30.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038362309999999998, 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:30.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991517+0000", 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2200000000000003e-07 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991539+0000", 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1583999999999999e-05 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991549+0000", 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0711e-05 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991554+0000", 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991587+0000", 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2495e-05 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991601+0000", 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4124000000000001e-05 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995329+0000", 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037274949999999999 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995353+0000", 2024-09-18T01:07:30.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4122e-05 2024-09-18T01:07:30.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.712 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:88 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'58, mlcod=16'51)", 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.689490304, 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032641620000000001, 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992329+0000", 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2099999999999996e-07 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992334+0000", 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0690000000000001e-06 2024-09-18T01:07:30.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992344+0000", 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8300000000000008e-06 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992349+0000", 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8879999999999996e-06 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995033+0000", 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026849360000000002 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995047+0000", 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3229e-05 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995573+0000", 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052584699999999997 2024-09-18T01:07:30.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995592+0000", 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9641999999999999e-05 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.721 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.568541642, 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002888641, 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8599999999999998e-07 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113345+0000", 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7858e-05 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113352+0000", 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113356+0000", 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2289999999999997e-06 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113421+0000", 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4591999999999994e-05 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113432+0000", 2024-09-18T01:07:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1499e-05 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116133+0000", 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027007110000000002 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116165+0000", 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2243000000000003e-05 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.568053373000001, 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027986180000000001, 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113772+0000", 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5540000000000003e-06 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113778+0000", 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5309999999999998e-06 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113792+0000", 2024-09-18T01:07:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3057999999999999e-05 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113834+0000", 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2749e-05 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113856+0000", 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1829000000000001e-05 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116534+0000", 2024-09-18T01:07:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026779769999999998 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116564+0000", 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9458000000000002e-05 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.557672894, 2024-09-18T01:07:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028065189999999999, 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124146+0000", 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9900000000000001e-07 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124209+0000", 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2821e-05 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124217+0000", 2024-09-18T01:07:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1270000000000003e-06 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124224+0000", 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1829999999999997e-06 2024-09-18T01:07:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124251+0000", 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7022e-05 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124262+0000", 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1175999999999999e-05 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.126924+0000", 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026615760000000001 2024-09-18T01:07:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.126952+0000", 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8215e-05 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:122 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'84, mlcod=16'82)", 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.127474+0000", 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.554344512, 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024743299999999998, 2024-09-18T01:07:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127474+0000", 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127474+0000", 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127474+0000", 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127477+0000", 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5950000000000001e-06 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127484+0000", 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7579999999999992e-06 2024-09-18T01:07:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127488+0000", 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127538+0000", 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0615999999999998e-05 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.127557+0000", 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8173999999999999e-05 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.129908+0000", 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023512770000000001 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.129948+0000", 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0203000000000001e-05 2024-09-18T01:07:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.174146546999999, 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034009729999999998, 2024-09-18T01:07:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507714+0000", 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1433000000000001e-05 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507742+0000", 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8283999999999999e-05 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507764+0000", 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2050000000000001e-05 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507844+0000", 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0313000000000002e-05 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507862+0000", 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7646999999999999e-05 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511043+0000", 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031807329999999998 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511073+0000", 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9876e-05 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.508395+0000", 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.173423811999999, 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027005330000000002, 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508395+0000", 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508395+0000", 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508395+0000", 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8400000000000005e-07 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508399+0000", 2024-09-18T01:07:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3590000000000001e-06 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508408+0000", 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9530000000000005e-06 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508415+0000", 2024-09-18T01:07:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508451+0000", 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000001e-05 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.508464+0000", 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.272e-05 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511074+0000", 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002609994 2024-09-18T01:07:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511095+0000", 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0985e-05 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.168629157000002, 2024-09-18T01:07:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044561130000000003, 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513190+0000", 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2900000000000004e-07 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513260+0000", 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0057999999999999e-05 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513268+0000", 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8900000000000007e-06 2024-09-18T01:07:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513275+0000", 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4120000000000004e-06 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513304+0000", 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8915999999999999e-05 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513315+0000", 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517614+0000", 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042987609999999999 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517645+0000", 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1253e-05 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'93)", 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.168075987999998, 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041452820000000001, 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513743+0000", 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513747+0000", 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5369999999999999e-06 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513753+0000", 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1149999999999999e-06 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513757+0000", 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2150000000000001e-06 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.515141+0000", 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013848510000000001 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.515153+0000", 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1579e-05 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517863+0000", 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002710114 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517888+0000", 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4561000000000001e-05 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-09-18T01:07:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.696896026000001, 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030854020000000001, 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984923+0000", 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8000000000000002e-07 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984926+0000", 2024-09-18T01:07:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2590000000000001e-06 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984932+0000", 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0079999999999999e-06 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984935+0000", 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2200000000000001e-06 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984970+0000", 2024-09-18T01:07:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4635000000000003e-05 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984983+0000", 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2947e-05 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987971+0000", 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002988746 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988008+0000", 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6307000000000003e-05 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.696369913000002, 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053365019999999999, 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985449+0000", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985533+0000", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4018999999999996e-05 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985539+0000", 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5549999999999999e-06 2024-09-18T01:07:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985542+0000", 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1370000000000002e-06 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985573+0000", 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0459000000000001e-05 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985585+0000", 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2442999999999999e-05 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990755+0000", 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051700569999999996 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990785+0000", 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9448999999999999e-05 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:156 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=16'105)", 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.695668605000002, 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046525639999999997, 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:07:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986157+0000", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3720000000000001e-06 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986167+0000", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9189999999999994e-06 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986171+0000", 2024-09-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5800000000000002e-06 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986205+0000", 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3587999999999998e-05 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986217+0000", 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2067e-05 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990787+0000", 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045698759999999996 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990802+0000", 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5509e-05 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:157 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'106)", 2024-09-18T01:07:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.695110383999999, 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004108464, 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986709+0000", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986713+0000", 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.639e-06 2024-09-18T01:07:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986723+0000", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9510000000000001e-06 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986728+0000", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4870000000000004e-06 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986773+0000", 2024-09-18T01:07:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451999999999998e-05 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986785+0000", 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1668000000000001e-05 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990803+0000", 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004018181 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990816+0000", 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3433e-05 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'106)", 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.694549704, 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035610030000000001, 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987335+0000", 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6458999999999995e-05 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987342+0000", 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1709999999999999e-06 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987345+0000", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.49e-06 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987373+0000", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7742999999999998e-05 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987384+0000", 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0866999999999999e-05 2024-09-18T01:07:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990817+0000", 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034332680000000002 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990830+0000", 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2670000000000001e-05 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:159 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'106)", 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.693825642, 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003105217, 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987997+0000", 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988002+0000", 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2070000000000003e-06 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988005+0000", 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988035+0000", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9757e-05 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988052+0000", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7782999999999999e-05 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.991075+0000", 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030231070000000001 2024-09-18T01:07:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.991098+0000", 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2478999999999999e-05 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:30.817 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:30.819 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:30.819 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:30.817+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:30.919 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:30.918+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:30.962 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:30.962 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:30.962 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:30.962 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:30.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:30.974 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:07:31.032 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:31.031+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:31.082 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:31.083 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:31.083 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:31.083 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:31.083 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:31.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:31.133 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:31.132+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:31.223 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:31.222+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:31.236 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.298371487000001, 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010922519999999999, 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937810+0000", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.11e-07 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937819+0000", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4069999999999999e-06 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937831+0000", 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1717e-05 2024-09-18T01:07:31.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937839+0000", 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4700000000000002e-06 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937900+0000", 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1008000000000003e-05 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937934+0000", 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3930999999999998e-05 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.938845+0000", 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00091077499999999995 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.938902+0000", 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6733e-05 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.283391098999999, 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00096652700000000001, 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2699999999999999e-07 2024-09-18T01:07:31.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952800+0000", 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952830+0000", 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0054999999999999e-05 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952854+0000", 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4219999999999999e-05 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952968+0000", 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113881 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952995+0000", 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6823000000000001e-05 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.953710+0000", 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071533399999999998 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.953756+0000", 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6227000000000003e-05 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.262577045, 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00093747999999999998, 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973609+0000", 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973623+0000", 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3491e-05 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973632+0000", 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4010000000000003e-06 2024-09-18T01:07:31.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973689+0000", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6589000000000003e-05 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973705+0000", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6222000000000002e-05 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.974477+0000", 2024-09-18T01:07:31.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000771738 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.974541+0000", 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4667999999999998e-05 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.25883149700000002, 2024-09-18T01:07:31.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00093139999999999998, 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977350+0000", 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.22e-07 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977358+0000", 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8460000000000008e-06 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977372+0000", 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4367e-05 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977382+0000", 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6369999999999994e-06 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977435+0000", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3421000000000002e-05 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977452+0000", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6855999999999999e-05 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.978230+0000", 2024-09-18T01:07:31.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077807099999999997 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.978281+0000", 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0880000000000001e-05 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:12 3.6 e26/23 3:655372e4:::rbd_header.1176f5693ee0:head v 26'1, mlcod=0'0)", 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:31.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.235551699, 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069200100000000003, 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7599999999999998e-07 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000635+0000", 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1170000000000003e-06 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000649+0000", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4052e-05 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000661+0000", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2075e-05 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000716+0000", 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5071000000000002e-05 2024-09-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000743+0000", 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7455999999999999e-05 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.001276+0000", 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053319299999999995 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.001321+0000", 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4761000000000001e-05 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:25 3.6 e26/23 3:655372e4:::rbd_header.1176f5693ee0:head v 26'2, mlcod=0'0)", 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.22265683, 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00060644599999999996, 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:31.264 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:31.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013526+0000", 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.624e-06 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013544+0000", 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8085999999999999e-05 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013560+0000", 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5962000000000001e-05 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013569+0000", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4870000000000008e-06 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013628+0000", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8869000000000002e-05 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013649+0000", 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0479000000000002e-05 2024-09-18T01:07:31.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.014095+0000", 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044690700000000002 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.014130+0000", 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5031999999999999e-05 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:31.269 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:31.269 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:31.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:31.323 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:31.322+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:31.373 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:31.373 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:31.382 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:31.425 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:31.423+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:31.508 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:31.525 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:31.524+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:31.528 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:31.528 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:31.528 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:31.528 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:31.528 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:31.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:31.626 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:31.625+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:31.648 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:31.649 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:31.680 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:31.680 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:31.712 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:31.712 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:31.720 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:31.720 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:31.726 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:31.725+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:31.731 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:31.731 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:31.732 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:31.747 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:31.747 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:31.765 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:31.765 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:31.783 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:31.783 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:31.800 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:31.801 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:31.827 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:31.826+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:31.852 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:31.853 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:31.861 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:31.946 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:31.945+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:32.007 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.008 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.017 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:32.028 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:32.027+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:32.129 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:32.129+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:32.166 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.166 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.170 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:32.230 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:32.229+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:32.285 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.286 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.294 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:32.336 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:32.335+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:32.432 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:32.431+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:32.438 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.438 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.447 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:32.533 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:32.532+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:32.590 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.590 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:32.639 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:32.638+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:32.693 INFO:teuthology.orchestra.run.smithi120.stdout:1 write 0x210402e thru 0x210692f (0x2902 bytes) 2024-09-18T01:07:32.697 INFO:teuthology.orchestra.run.smithi120.stdout:3 read 0x9f4ed2 thru 0xa00af3 (0xbc22 bytes) 2024-09-18T01:07:32.697 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.711 INFO:teuthology.orchestra.run.smithi120.stdout:7 writesame 0xa0c35ea thru 0xa0c681a data_size 0x1(0x3231 bytes) 2024-09-18T01:07:32.711 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xb3f77c9 2024-09-18T01:07:32.712 INFO:teuthology.orchestra.run.smithi120.stdout:9 trunc from 0xa0c681b to 0xb3f77c9 2024-09-18T01:07:32.721 INFO:teuthology.orchestra.run.smithi120.stdout:10 read 0x7b4e266 thru 0x7b4f8dd (0x1678 bytes) 2024-09-18T01:07:32.721 INFO:teuthology.orchestra.run.smithi120.stdout:11 read 0xa42d4f5 thru 0xa42ecf8 (0x1804 bytes) 2024-09-18T01:07:32.721 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.721 INFO:teuthology.orchestra.run.smithi120.stdout:14 write 0xfed738 thru 0xfef1e8 (0x1ab1 bytes) 2024-09-18T01:07:32.724 INFO:teuthology.orchestra.run.smithi120.stdout:15 write 0x66f814 thru 0x674cd8 (0x54c5 bytes) 2024-09-18T01:07:32.727 INFO:teuthology.orchestra.run.smithi120.stdout:17 read 0x8e06e28 thru 0x8e13ae3 (0xccbc bytes) 2024-09-18T01:07:32.727 INFO:teuthology.orchestra.run.smithi120.stdout:18 writesame 0x6d600af thru 0x6d65565 data_size 0x1(0x54b7 bytes) 2024-09-18T01:07:32.734 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:32.733+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:32.740 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.740 INFO:teuthology.orchestra.run.smithi120.stdout:21 writesame 0x9ffe73d thru 0xa0053d0 data_size 0x2(0x6c94 bytes) 2024-09-18T01:07:32.748 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.748 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.758 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:32.760 INFO:teuthology.orchestra.run.smithi120.stdout:24 write 0xe840fd6 thru 0xe84b9d2 (0xa9fd bytes) 2024-09-18T01:07:32.763 INFO:teuthology.orchestra.run.smithi120.stdout:25 writesame 0x2f10953 thru 0x2f1c3dd data_size 0x1(0xba8b bytes) 2024-09-18T01:07:32.788 INFO:teuthology.orchestra.run.smithi120.stdout:26 write 0x364d265 thru 0x365b9ed (0xe789 bytes) 2024-09-18T01:07:32.791 INFO:teuthology.orchestra.run.smithi120.stdout:30 write 0x99a281a thru 0x99ae7b1 (0xbf98 bytes) 2024-09-18T01:07:32.809 INFO:teuthology.orchestra.run.smithi120.stdout:31 trunc from 0xe84b9d3 to 0x1f4c0af 2024-09-18T01:07:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:32 writesame 0x6043e3a thru 0x604e1ea data_size 0x1d(0xa3b1 bytes) 2024-09-18T01:07:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:33 read 0x39686c6 thru 0x396ad1c (0x2657 bytes) 2024-09-18T01:07:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:34 read 0x479bc thru 0x48129 (0x76e bytes) 2024-09-18T01:07:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:37 read 0x233ad62 thru 0x2348073 (0xd312 bytes) 2024-09-18T01:07:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:38 punch from 0x21564e8 to 0x215fb61, (0x9679 bytes) 2024-09-18T01:07:32.833 INFO:teuthology.orchestra.run.smithi120.stdout:40 read 0x38ff2d5 thru 0x3905c85 (0x69b1 bytes) 2024-09-18T01:07:32.843 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:32.842+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:32.844 INFO:teuthology.orchestra.run.smithi120.stdout:42 write 0xc0218ee thru 0xc0222bf (0x9d2 bytes) 2024-09-18T01:07:32.847 INFO:teuthology.orchestra.run.smithi120.stdout:43 write 0x5ab896c thru 0x5abac87 (0x231c bytes) 2024-09-18T01:07:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:44 read 0x6fc1999 thru 0x6fc4274 (0x28dc bytes) 2024-09-18T01:07:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:45 writesame 0xb942999 thru 0xb9457c1 data_size 0x27(0x2e29 bytes) 2024-09-18T01:07:32.853 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xc93ce71 2024-09-18T01:07:32.853 INFO:teuthology.orchestra.run.smithi120.stdout:46 trunc from 0xc0222c0 to 0xc93ce71 2024-09-18T01:07:32.858 INFO:teuthology.orchestra.run.smithi120.stdout:47 read 0xadbd459 thru 0xadbfd27 (0x28cf bytes) 2024-09-18T01:07:32.864 INFO:teuthology.orchestra.run.smithi120.stdout:48 trunc from 0xc93ce71 to 0x811fbb7 2024-09-18T01:07:32.880 INFO:teuthology.orchestra.run.smithi120.stdout:49 trunc from 0x811fbb7 to 0x1918b30 2024-09-18T01:07:32.906 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:32.907 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:32.908 INFO:teuthology.orchestra.run.smithi120.stdout:50 write 0xcf6fab8 thru 0xcf7ce0b (0xd354 bytes) 2024-09-18T01:07:32.911 INFO:teuthology.orchestra.run.smithi120.stdout:56 read 0x7cecc47 thru 0x7cf83ec (0xb7a6 bytes) 2024-09-18T01:07:32.911 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.911 INFO:teuthology.orchestra.run.smithi120.stdout:61 punch from 0xb4e7a73 to 0xb4f21fd, (0xa78a bytes) 2024-09-18T01:07:32.911 INFO:teuthology.orchestra.run.smithi120.stdout:62 writesame 0x4783e64 thru 0x47870b2 data_size 0x9(0x324f bytes) 2024-09-18T01:07:32.914 INFO:teuthology.orchestra.run.smithi120.stdout:64 write 0x64ee0fd thru 0x64f1901 (0x3805 bytes) 2024-09-18T01:07:32.916 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:32.917 INFO:teuthology.orchestra.run.smithi120.stdout:66 punch from 0x601e7a0 to 0x602658f, (0x7def bytes) 2024-09-18T01:07:32.918 INFO:teuthology.orchestra.run.smithi120.stdout:68 write 0x1289acb thru 0x1297f10 (0xe446 bytes) 2024-09-18T01:07:32.921 INFO:teuthology.orchestra.run.smithi120.stdout:69 write 0x2de7c83 thru 0x2df1d00 (0xa07e bytes) 2024-09-18T01:07:32.923 INFO:teuthology.orchestra.run.smithi120.stdout:70 write 0x60bc48d thru 0x60c2f60 (0x6ad4 bytes) 2024-09-18T01:07:32.926 INFO:teuthology.orchestra.run.smithi120.stdout:71 read 0x5aeffca thru 0x5af915d (0x9194 bytes) 2024-09-18T01:07:32.926 INFO:teuthology.orchestra.run.smithi120.stdout:73 writesame 0x272ff16 thru 0x273c4a3 data_size 0x3(0xc58e bytes) 2024-09-18T01:07:32.936 INFO:teuthology.orchestra.run.smithi120.stdout:76 write 0xbbf724b thru 0xbc02c18 (0xb9ce bytes) 2024-09-18T01:07:32.940 INFO:teuthology.orchestra.run.smithi120.stdout:77 punch from 0x4531a78 to 0x4541176, (0xf6fe bytes) 2024-09-18T01:07:32.940 INFO:teuthology.orchestra.run.smithi120.stdout:78 writesame 0x5653a thru 0x5a731 data_size 0x4(0x41f8 bytes) 2024-09-18T01:07:32.944 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.944 INFO:teuthology.orchestra.run.smithi120.stdout:81 punch from 0x161d970 to 0x1623841, (0x5ed1 bytes) 2024-09-18T01:07:32.944 INFO:teuthology.orchestra.run.smithi120.stdout:82 read 0x5d4fc46 thru 0x5d5c3f5 (0xc7b0 bytes) 2024-09-18T01:07:32.944 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.944 INFO:teuthology.orchestra.run.smithi120.stdout:84 punch from 0xb05f790 to 0xb068912, (0x9182 bytes) 2024-09-18T01:07:32.951 INFO:teuthology.orchestra.run.smithi120.stdout:85 write 0xeaaf805 thru 0xeabb30a (0xbb06 bytes) 2024-09-18T01:07:32.955 INFO:teuthology.orchestra.run.smithi120.stdout:86 write 0x4daa078 thru 0x4db985d (0xf7e6 bytes) 2024-09-18T01:07:32.958 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:32.956+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:32.958 INFO:teuthology.orchestra.run.smithi120.stdout:87 write 0x258f9bf thru 0x2594f32 (0x5574 bytes) 2024-09-18T01:07:32.959 INFO:teuthology.orchestra.run.smithi120.stdout:88 write 0x347ee24 thru 0x347fb30 (0xd0d bytes) 2024-09-18T01:07:32.962 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.963 INFO:teuthology.orchestra.run.smithi120.stdout:90 writesame 0xd1c0236 thru 0xd1cbadc data_size 0x7(0xb8a7 bytes) 2024-09-18T01:07:32.968 INFO:teuthology.orchestra.run.smithi120.stdout:91 read 0x7b2480e thru 0x7b265be (0x1db1 bytes) 2024-09-18T01:07:32.968 INFO:teuthology.orchestra.run.smithi120.stdout:92 write 0x4004e63 thru 0x4010cef (0xbe8d bytes) 2024-09-18T01:07:32.972 INFO:teuthology.orchestra.run.smithi120.stdout:93 read 0x8e9a5ba thru 0x8e9cc64 (0x26ab bytes) 2024-09-18T01:07:32.972 INFO:teuthology.orchestra.run.smithi120.stdout:96 read 0xa099344 thru 0xa09a405 (0x10c2 bytes) 2024-09-18T01:07:32.972 INFO:teuthology.orchestra.run.smithi120.stdout:98 write 0xc889588 thru 0xc891b37 (0x85b0 bytes) 2024-09-18T01:07:32.977 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:32.978 INFO:teuthology.orchestra.run.smithi120.stdout:100 write 0x1c95b0b thru 0x1c9c21a (0x6710 bytes) 2024-09-18T01:07:32.981 INFO:teuthology.orchestra.run.smithi120.stdout:101 punch from 0x2409066 to 0x2412d3e, (0x9cd8 bytes) 2024-09-18T01:07:32.981 INFO:teuthology.orchestra.run.smithi120.stdout:102 punch from 0xb785a46 to 0xb790568, (0xab22 bytes) 2024-09-18T01:07:32.981 INFO:teuthology.orchestra.run.smithi120.stdout:103 writesame 0x3ad7156 thru 0x3ade9b7 data_size 0x13(0x7862 bytes) 2024-09-18T01:07:32.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:07:32.995 INFO:teuthology.orchestra.run.smithi120.stdout:104 trunc from 0xeabb30b to 0x7cf67ba 2024-09-18T01:07:33.010 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:33.011 INFO:teuthology.orchestra.run.smithi120.stdout:106 trunc from 0x7cf67ba to 0x7cbccde 2024-09-18T01:07:33.029 INFO:teuthology.orchestra.run.smithi120.stdout:107 write 0xeabfd9f thru 0xeac7572 (0x77d4 bytes) 2024-09-18T01:07:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:108 write 0xd377541 thru 0xd37812f (0xbef bytes) 2024-09-18T01:07:33.035 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:33.034+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:33.038 INFO:teuthology.orchestra.run.smithi120.stdout:109 read 0xdc5e044 thru 0xdc5ee6b (0xe28 bytes) 2024-09-18T01:07:33.038 INFO:teuthology.orchestra.run.smithi120.stdout:110 read 0xbd78401 thru 0xbd81bb3 (0x97b3 bytes) 2024-09-18T01:07:33.038 INFO:teuthology.orchestra.run.smithi120.stdout:112 punch from 0x3e3faff to 0x3e4891c, (0x8e1d bytes) 2024-09-18T01:07:33.038 INFO:teuthology.orchestra.run.smithi120.stdout:113 punch from 0x41c59fc to 0x41d294a, (0xcf4e bytes) 2024-09-18T01:07:33.039 INFO:teuthology.orchestra.run.smithi120.stdout:114 punch from 0xc233b29 to 0xc23ff9d, (0xc474 bytes) 2024-09-18T01:07:33.039 INFO:teuthology.orchestra.run.smithi120.stdout:117 punch from 0x8ccef66 to 0x8cd7d80, (0x8e1a bytes) 2024-09-18T01:07:33.041 INFO:teuthology.orchestra.run.smithi120.stdout:119 trunc from 0xeac7573 to 0xb9e4348 2024-09-18T01:07:33.057 INFO:teuthology.orchestra.run.smithi120.stdout:120 trunc from 0xb9e4348 to 0xb0cd179 2024-09-18T01:07:33.062 INFO:teuthology.orchestra.run.smithi120.stdout:121 writesame 0x295de60 thru 0x2961a35 data_size 0x17(0x3bd6 bytes) 2024-09-18T01:07:33.065 INFO:teuthology.orchestra.run.smithi120.stdout:122 read 0x272e3b9 thru 0x2739662 (0xb2aa bytes) 2024-09-18T01:07:33.066 INFO:teuthology.orchestra.run.smithi120.stdout:123 write 0x4409458 thru 0x440e03a (0x4be3 bytes) 2024-09-18T01:07:33.067 INFO:teuthology.orchestra.run.smithi120.stdout:124 write 0xad0f1fd thru 0xad125c3 (0x33c7 bytes) 2024-09-18T01:07:33.067 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.067 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.070 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:33.070 INFO:teuthology.orchestra.run.smithi120.stdout:126 writesame 0x91dc4ca thru 0x91e35e7 data_size 0x1(0x711e bytes) 2024-09-18T01:07:33.078 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:33.095 INFO:teuthology.orchestra.run.smithi120.stdout:129 write 0xe8df1db thru 0xe8e66cc (0x74f2 bytes) 2024-09-18T01:07:33.099 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:33.099 INFO:teuthology.orchestra.run.smithi120.stdout:131 read 0x7500a9d thru 0x750151f (0xa83 bytes) 2024-09-18T01:07:33.099 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:33.100 INFO:teuthology.orchestra.run.smithi120.stdout:133 writesame 0xb2bdccd thru 0xb2ca86d data_size 0xb(0xcba1 bytes) 2024-09-18T01:07:33.106 INFO:teuthology.orchestra.run.smithi120.stdout:135 clone 1 order 20 su 16384 sc 8 2024-09-18T01:07:33.136 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:33.135+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:33.221 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.222 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:33.237 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:33.236+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:33.333 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.333 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.339 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:33.338+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:33.342 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:33.440 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:33.439+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:33.444 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.444 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:33.541 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:33.540+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:33.557 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.557 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.566 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:33.650 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:33.649+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:33.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:07:33.659 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:07:33.710 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.710 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.719 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:33.742 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:33.742+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:33.813 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:33.857 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:33.856+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:33.862 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:33.862 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:33.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:33.936 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:33.936 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:33.945 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:33.944+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:33.953 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:33.970 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:33.979 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:33.979 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:33.986 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:33.987 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:33.988 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:33.993 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:07:33.993 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:07:34.010 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:34.011 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:34.014 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:07:34.030 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:34.030 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:34.045 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:34.044+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:34.048 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:34.048 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:34.067 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:34.067 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:34.104 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:34.104 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:34.146 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:34.145+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:34.246 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:34.245+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:34.258 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.267590941999998, 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005693783, 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989265+0000", 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989266+0000", 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989270+0000", 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7190000000000001e-06 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989281+0000", 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.063e-05 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989286+0000", 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989323+0000", 2024-09-18T01:07:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7370000000000003e-05 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989351+0000", 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7874000000000001e-05 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994914+0000", 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055624790000000004 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994959+0000", 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5194999999999997e-05 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.267101703000002, 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052255940000000001, 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.264 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989754+0000", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989761+0000", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8539999999999999e-06 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989821+0000", 2024-09-18T01:07:34.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0129e-05 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989831+0000", 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4679999999999998e-06 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989835+0000", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989865+0000", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0494999999999999e-05 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.989877+0000", 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.201e-05 2024-09-18T01:07:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994961+0000", 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005083873 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994980+0000", 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8675999999999999e-05 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.269 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:85 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.266466076, 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046080890000000001, 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:34.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990390+0000", 2024-09-18T01:07:34.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990394+0000", 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6389999999999998e-06 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990401+0000", 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6320000000000002e-06 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990404+0000", 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990436+0000", 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1869999999999998e-05 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990449+0000", 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3097e-05 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994982+0000", 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004532683 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.994998+0000", 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6056e-05 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'51)", 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:34.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.265938547000001, 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040959600000000001, 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990917+0000", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990918+0000", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990927+0000", 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5959999999999993e-06 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990934+0000", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3430000000000001e-06 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990937+0000", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6789999999999998e-06 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990966+0000", 2024-09-18T01:07:34.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8986e-05 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.990979+0000", 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2351000000000001e-05 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995000+0000", 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040208450000000003 2024-09-18T01:07:34.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995013+0000", 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3773e-05 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.280 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'51)", 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.265339535999999, 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038362309999999998, 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991516+0000", 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991517+0000", 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2200000000000003e-07 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991539+0000", 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1583999999999999e-05 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991549+0000", 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0711e-05 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991554+0000", 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991587+0000", 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2495e-05 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.991601+0000", 2024-09-18T01:07:34.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4124000000000001e-05 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995329+0000", 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037274949999999999 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995353+0000", 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4122e-05 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.285 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:88 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'58, mlcod=16'51)", 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.264527984000001, 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032641620000000001, 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992328+0000", 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992329+0000", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2099999999999996e-07 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992334+0000", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0690000000000001e-06 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992344+0000", 2024-09-18T01:07:34.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8300000000000008e-06 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.992349+0000", 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8879999999999996e-06 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995033+0000", 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026849360000000002 2024-09-18T01:07:34.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995047+0000", 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3229e-05 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995573+0000", 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052584699999999997 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:12.995592+0000", 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9641999999999999e-05 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.143579322000001, 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002888641, 2024-09-18T01:07:34.291 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113277+0000", 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8599999999999998e-07 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113345+0000", 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7858e-05 2024-09-18T01:07:34.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113352+0000", 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113356+0000", 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2289999999999997e-06 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113421+0000", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4591999999999994e-05 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113432+0000", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1499e-05 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116133+0000", 2024-09-18T01:07:34.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027007110000000002 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116165+0000", 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2243000000000003e-05 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.296 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.143091052999999, 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027986180000000001, 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:34.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113765+0000", 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113772+0000", 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5540000000000003e-06 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113778+0000", 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5309999999999998e-06 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113792+0000", 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3057999999999999e-05 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113834+0000", 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2749e-05 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.113856+0000", 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1829000000000001e-05 2024-09-18T01:07:34.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116534+0000", 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026779769999999998 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.116564+0000", 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9458000000000002e-05 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.301 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.132710574000001, 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028065189999999999, 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124145+0000", 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124146+0000", 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9900000000000001e-07 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124209+0000", 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2821e-05 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124217+0000", 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1270000000000003e-06 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124224+0000", 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1829999999999997e-06 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124251+0000", 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7022e-05 2024-09-18T01:07:34.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.124262+0000", 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1175999999999999e-05 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.126924+0000", 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026615760000000001 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.126952+0000", 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8215e-05 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.749184227000001, 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034009729999999998, 2024-09-18T01:07:34.307 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:34.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507672+0000", 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507714+0000", 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1433000000000001e-05 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507742+0000", 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8283999999999999e-05 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507764+0000", 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2050000000000001e-05 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507844+0000", 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0313000000000002e-05 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.507862+0000", 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7646999999999999e-05 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511043+0000", 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031807329999999998 2024-09-18T01:07:34.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.511073+0000", 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9876e-05 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-18T01:07:34.312 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.743666836999999, 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044561130000000003, 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513189+0000", 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513190+0000", 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2900000000000004e-07 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513260+0000", 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0057999999999999e-05 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513268+0000", 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8900000000000007e-06 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513275+0000", 2024-09-18T01:07:34.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4120000000000004e-06 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513304+0000", 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8915999999999999e-05 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513315+0000", 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517614+0000", 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042987609999999999 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517645+0000", 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1253e-05 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'93)", 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.743113667999999, 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041452820000000001, 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.318 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513742+0000", 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513743+0000", 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513747+0000", 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5369999999999999e-06 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513753+0000", 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1149999999999999e-06 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.513757+0000", 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2150000000000001e-06 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.515141+0000", 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013848510000000001 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.515153+0000", 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1579e-05 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517863+0000", 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002710114 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.517888+0000", 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4561000000000001e-05 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.271933705999999, 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030854020000000001, 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984922+0000", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984923+0000", 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8000000000000002e-07 2024-09-18T01:07:34.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984926+0000", 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2590000000000001e-06 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984932+0000", 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0079999999999999e-06 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984935+0000", 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2200000000000001e-06 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984970+0000", 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4635000000000003e-05 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.984983+0000", 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2947e-05 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987971+0000", 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002988746 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988008+0000", 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6307000000000003e-05 2024-09-18T01:07:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.271407592999999, 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053365019999999999, 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985448+0000", 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985449+0000", 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985533+0000", 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4018999999999996e-05 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985539+0000", 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5549999999999999e-06 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985542+0000", 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1370000000000002e-06 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985573+0000", 2024-09-18T01:07:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0459000000000001e-05 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.985585+0000", 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2442999999999999e-05 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990755+0000", 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051700569999999996 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990785+0000", 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9448999999999999e-05 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:156 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=16'105)", 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:34.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.270706284999999, 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046525639999999997, 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986150+0000", 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986157+0000", 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3720000000000001e-06 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986167+0000", 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9189999999999994e-06 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986171+0000", 2024-09-18T01:07:34.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5800000000000002e-06 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986205+0000", 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3587999999999998e-05 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986217+0000", 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2067e-05 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990787+0000", 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045698759999999996 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990802+0000", 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5509e-05 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:157 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'106)", 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.270148064000001, 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004108464, 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986708+0000", 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986709+0000", 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986713+0000", 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.639e-06 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986723+0000", 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9510000000000001e-06 2024-09-18T01:07:34.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986728+0000", 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4870000000000004e-06 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986773+0000", 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451999999999998e-05 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.986785+0000", 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1668000000000001e-05 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990803+0000", 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004018181 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990816+0000", 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3433e-05 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.345 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'106)", 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.269587384000001, 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035610030000000001, 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987269+0000", 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987335+0000", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6458999999999995e-05 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987342+0000", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1709999999999999e-06 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987345+0000", 2024-09-18T01:07:34.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.49e-06 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987373+0000", 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7742999999999998e-05 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987384+0000", 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0866999999999999e-05 2024-09-18T01:07:34.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990817+0000", 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034332680000000002 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.990830+0000", 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2670000000000001e-05 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4234.0:159 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'106)", 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.268863322000001, 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003105217, 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987993+0000", 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.987997+0000", 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988002+0000", 2024-09-18T01:07:34.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2070000000000003e-06 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988005+0000", 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988035+0000", 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9757e-05 2024-09-18T01:07:34.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.988052+0000", 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7782999999999999e-05 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.991075+0000", 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030231070000000001 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:13.991098+0000", 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2478999999999999e-05 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:30.994787+0000", 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.2620686989999998, 2024-09-18T01:07:34.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053389099999999997, 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 11 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994787+0000", 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994787+0000", 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994789+0000", 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8160000000000001e-06 2024-09-18T01:07:34.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994800+0000", 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0496000000000001e-05 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994817+0000", 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7652999999999999e-05 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994826+0000", 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7790000000000001e-06 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994883+0000", 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6833999999999998e-05 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.994891+0000", 2024-09-18T01:07:34.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.996678+0000", 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001787486 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.996778+0000", 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100271 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.996963+0000", 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000185179 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.998027+0000", 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001063123 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000025+0000", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019987329999999999 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000081+0000", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754000000000002e-05 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000126+0000", 2024-09-18T01:07:34.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5139999999999998e-05 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:31.006194+0000", 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.2506615729999999, 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030304099999999999, 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 15 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006194+0000", 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006194+0000", 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006195+0000", 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:07:34.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006199+0000", 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5470000000000001e-06 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006206+0000", 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6760000000000007e-06 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006211+0000", 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2560000000000004e-06 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006240+0000", 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9332999999999999e-05 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.006282+0000", 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2398999999999998e-05 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.009225+0000", 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002942592 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.370 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.370 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:34.372 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:34.370+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:34.472 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:34.471+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:34.514 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.515 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:34.515 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:34.515 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.523 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:34.573 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:34.571+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:34.625 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:34.625 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.634 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:34.674 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:34.673+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:34.738 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.799243188, 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010922519999999999, 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937809+0000", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937810+0000", 2024-09-18T01:07:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.11e-07 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937819+0000", 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4069999999999999e-06 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937831+0000", 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1717e-05 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937839+0000", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4700000000000002e-06 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937900+0000", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1008000000000003e-05 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.937934+0000", 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3930999999999998e-05 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.938845+0000", 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00091077499999999995 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:19.938902+0000", 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6733e-05 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.7842628, 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00096652700000000001, 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952790+0000", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2699999999999999e-07 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952800+0000", 2024-09-18T01:07:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952830+0000", 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0054999999999999e-05 2024-09-18T01:07:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952854+0000", 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4219999999999999e-05 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952968+0000", 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113881 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.952995+0000", 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6823000000000001e-05 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.953710+0000", 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071533399999999998 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:20.953756+0000", 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6227000000000003e-05 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.7634487459999999, 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00093747999999999998, 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973604+0000", 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973609+0000", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973623+0000", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3491e-05 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973632+0000", 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4010000000000003e-06 2024-09-18T01:07:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973689+0000", 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6589000000000003e-05 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.973705+0000", 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6222000000000002e-05 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.974477+0000", 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000771738 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:29.974541+0000", 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4667999999999998e-05 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.759703198, 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00093139999999999998, 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977349+0000", 2024-09-18T01:07:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977350+0000", 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.22e-07 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977358+0000", 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8460000000000008e-06 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977372+0000", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4367e-05 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977382+0000", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6369999999999994e-06 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977435+0000", 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3421000000000002e-05 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.977452+0000", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6855999999999999e-05 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.978230+0000", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077807099999999997 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:30.978281+0000", 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0880000000000001e-05 2024-09-18T01:07:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:12 3.6 e26/23 3:655372e4:::rbd_header.1176f5693ee0:head v 26'1, mlcod=0'0)", 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7364234000000001, 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069200100000000003, 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000629+0000", 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7599999999999998e-07 2024-09-18T01:07:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000635+0000", 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1170000000000003e-06 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000649+0000", 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4052e-05 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000661+0000", 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2075e-05 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000716+0000", 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5071000000000002e-05 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.000743+0000", 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7455999999999999e-05 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.001276+0000", 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053319299999999995 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.001321+0000", 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4761000000000001e-05 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:25 3.6 e26/23 3:655372e4:::rbd_header.1176f5693ee0:head v 26'2, mlcod=0'0)", 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7235285309999999, 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00060644599999999996, 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013524+0000", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013526+0000", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.624e-06 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013544+0000", 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8085999999999999e-05 2024-09-18T01:07:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013560+0000", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5962000000000001e-05 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013569+0000", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4870000000000008e-06 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013628+0000", 2024-09-18T01:07:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8869000000000002e-05 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.013649+0000", 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0479000000000002e-05 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.014095+0000", 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044690700000000002 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:31.014130+0000", 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5031999999999999e-05 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:332 3.6 e27/23 3:655372e4:::rbd_header.1176f5693ee0:head v 27'26, mlcod=26'24)", 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:33.994012+0000", 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.74304070300000002, 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00080589699999999997, 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994012+0000", 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994012+0000", 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994012+0000", 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994021+0000", 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0999999999999993e-06 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994034+0000", 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.224e-05 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994044+0000", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0327e-05 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994109+0000", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4568999999999998e-05 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994133+0000", 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4947e-05 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994765+0000", 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00063171199999999999 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.994818+0000", 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2609999999999999e-05 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:334 3.0 3:03471435:::rbd_object_map.1176f5693ee0.0000000000000003:head [writefull 0~49] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:33.997486+0000", 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.73956620500000003, 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018905219999999999, 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 334 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997486+0000", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997486+0000", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997488+0000", 2024-09-18T01:07:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997499+0000", 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0428000000000001e-05 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997514+0000", 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5579000000000001e-05 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997524+0000", 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2900000000000008e-06 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997586+0000", 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2106999999999993e-05 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997694+0000", 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107815 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.997795+0000", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101562 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.998845+0000", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010502020000000001 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999268+0000", 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042287999999999999 2024-09-18T01:07:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999327+0000", 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8474000000000003e-05 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999377+0000", 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0225000000000001e-05 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop_reply(client.4470.0:334 3.0 e26/23 ondisk, result = 0)", 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:33.999164+0000", 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.73788843199999998, 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022236599999999999, 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999164+0000", 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999164+0000", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999165+0000", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.305e-06 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999172+0000", 2024-09-18T01:07:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999187+0000", 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5024e-05 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999196+0000", 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9080000000000001e-06 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999250+0000", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4381e-05 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999260+0000", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6700000000000006e-06 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:33.999387+0000", 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126365 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:346 3.6 e27/23 3:655372e4:::rbd_header.1176f5693ee0:head v 27'27, mlcod=26'25)", 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.006047+0000", 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.73100541200000002, 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069155799999999999, 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006047+0000", 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006047+0000", 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006049+0000", 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.35e-06 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006060+0000", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0995999999999999e-05 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006076+0000", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6779999999999999e-05 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006086+0000", 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6020000000000006e-06 2024-09-18T01:07:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006157+0000", 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1215000000000002e-05 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006184+0000", 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6613000000000001e-05 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006692+0000", 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050832999999999998 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.006739+0000", 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6672000000000003e-05 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:356 3.6 e27/23 3:655372e4:::rbd_header.1176f5693ee0:head v 27'28, mlcod=27'26)", 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.013685+0000", 2024-09-18T01:07:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.723367439, 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925599999999999, 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013685+0000", 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013685+0000", 2024-09-18T01:07:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013687+0000", 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013699+0000", 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2376e-05 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013717+0000", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7716000000000001e-05 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013726+0000", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0110000000000006e-06 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013805+0000", 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8751999999999994e-05 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.013827+0000", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2005999999999999e-05 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014272+0000", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044477999999999998 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014304+0000", 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2869e-05 2024-09-18T01:07:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:358 3.0 3:03471435:::rbd_object_map.1176f5693ee0.0000000000000003:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.014657+0000", 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.722395966, 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034208369999999999, 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 358 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014657+0000", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014657+0000", 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014658+0000", 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.767e-06 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014666+0000", 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4719999999999998e-06 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014681+0000", 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4678000000000001e-05 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014696+0000", 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5523999999999999e-05 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014788+0000", 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2393999999999997e-05 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.014887+0000", 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8966000000000001e-05 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.018077+0000", 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031900359999999998 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:366 3.1a e27/23 3:5e4f253b:::rbd_header.11769efc726f:head v 27'4, mlcod=26'2)", 2024-09-18T01:07:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.024833+0000", 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.71221922299999996, 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003292021, 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024833+0000", 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024833+0000", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024834+0000", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024840+0000", 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4949999999999996e-06 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024852+0000", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1627000000000001e-05 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024872+0000", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0058999999999999e-05 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024921+0000", 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8937000000000003e-05 2024-09-18T01:07:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.024936+0000", 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5475999999999999e-05 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028084+0000", 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031479609999999999 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028125+0000", 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1097000000000003e-05 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:367 3.1a e27/23 3:5e4f253b:::rbd_header.11769efc726f:head v 27'5, mlcod=26'3)", 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.028988+0000", 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.70806475199999996, 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052363799999999999, 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028988+0000", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028988+0000", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028989+0000", 2024-09-18T01:07:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4900000000000005e-07 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.028994+0000", 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029004+0000", 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9170000000000001e-06 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029009+0000", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029049+0000", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9978000000000002e-05 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029067+0000", 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-05 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029485+0000", 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00041803000000000001 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.029511+0000", 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6393999999999998e-05 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:381 3.1a e27/23 3:5e4f253b:::rbd_header.11769efc726f:head v 27'6, mlcod=27'4)", 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.038530+0000", 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.69852282399999999, 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00075792100000000005, 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038530+0000", 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038530+0000", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038531+0000", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038542+0000", 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0529e-05 2024-09-18T01:07:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038559+0000", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7615999999999999e-05 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038568+0000", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038626+0000", 2024-09-18T01:07:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7985999999999997e-05 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.038647+0000", 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1821999999999999e-05 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.039238+0000", 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00059080999999999999 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.039288+0000", 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9471000000000001e-05 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:383 3.1a e27/23 3:5e4f253b:::rbd_header.11769efc726f:head v 27'7, mlcod=27'5)", 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:34.040275+0000", 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.696777969, 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018084329999999999, 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040275+0000", 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040275+0000", 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040276+0000", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1400000000000001e-06 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040284+0000", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2910000000000007e-06 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040297+0000", 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3451999999999999e-05 2024-09-18T01:07:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040306+0000", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4079999999999998e-06 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040354+0000", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8260000000000002e-05 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.040369+0000", 2024-09-18T01:07:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5248e-05 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.042040+0000", 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016707779999999999 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:34.042083+0000", 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2855999999999998e-05 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:34.814 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.815 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:34.815 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:34.814+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:34.916 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:34.915+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:34.959 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:34.959 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:34.959 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:34.959 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:34.968 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:35.002 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:07:35.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:07:35.020 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:35.018+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:35.118 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:35.117+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:35.162 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:35.162 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:35.162 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:35.162 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:35.162 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:35.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:35.219 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:35.218+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:35.320 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:35.319+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:35.333 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:35.333 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:35.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:35.333 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.391765760999998, 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:07:35.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:783 3.0 e28/23 3:04cd6778:::rbd_data.11769efc726f.0000000000000005:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:35.338 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.28557463500000002, 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026120100000000001, 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045917+0000", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0500000000000003e-07 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046106+0000", 2024-09-18T01:07:35.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018893600000000001 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046125+0000", 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9128e-05 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046137+0000", 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046199+0000", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2520000000000004e-05 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046222+0000", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2554000000000001e-05 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048480+0000", 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002258047 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048528+0000", 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8241000000000001e-05 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:870 3.0 e28/23 3:001618d7:::rbd_data.11769efc726f.000000000000000b:head v 28'3, mlcod=27'1)", 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.27182584199999998, 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019550090000000002, 2024-09-18T01:07:35.343 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:35.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059666+0000", 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.601e-06 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059981+0000", 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031502199999999998 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060002+0000", 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0237e-05 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060011+0000", 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3999999999999998e-06 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060085+0000", 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3720999999999994e-05 2024-09-18T01:07:35.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060112+0000", 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7019e-05 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061577+0000", 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014653559999999999 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061620+0000", 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2653e-05 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:871 3.0 e28/23 3:0320ffd9:::rbd_data.11769efc726f.0000000000000011:head v 28'4, mlcod=27'1)", 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:35.348 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.270403956, 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00170341, 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061088+0000", 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-06 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061674+0000", 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00058667599999999995 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061688+0000", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3709999999999999e-05 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061696+0000", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061761+0000", 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5508999999999999e-05 2024-09-18T01:07:35.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061785+0000", 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3407000000000001e-05 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.062748+0000", 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00096303600000000001 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.062790+0000", 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2348000000000002e-05 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.26383772100000003, 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:07:35.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1087 3.0 e28/23 3:06d5c3a2:::rbd_data.11769efc726f.000000000000000d:head v 28'6, mlcod=28'4)", 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.075713+0000", 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.25577772300000001, 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001678789, 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.075713+0000", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.075713+0000", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.075715+0000", 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:07:35.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.076018+0000", 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030334300000000003 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.076037+0000", 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8848999999999999e-05 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.076046+0000", 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7909999999999993e-06 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.076103+0000", 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7763999999999998e-05 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.076127+0000", 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3240000000000001e-05 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.077347+0000", 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001220304 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.077392+0000", 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4718000000000001e-05 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1145 3.0 e28/23 3:0368a151:::rbd_data.11769efc726f.000000000000001b:head v 28'7, mlcod=28'5)", 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.24000168999999999, 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026055610000000002, 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091490+0000", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091544+0000", 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3934999999999998e-05 2024-09-18T01:07:35.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091563+0000", 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8423e-05 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091576+0000", 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3010000000000001e-05 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091636+0000", 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0732000000000001e-05 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091658+0000", 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1943999999999999e-05 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094060+0000", 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00240136 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094095+0000", 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1144 3.9 e28/23 3:95bdb02d:::rbd_data.11769efc726f.0000000000000016:head v 28'1, mlcod=0'0)", 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.239562056, 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021202579999999999, 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092370+0000", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044077999999999999 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092382+0000", 2024-09-18T01:07:35.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2649999999999999e-05 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092391+0000", 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2090000000000005e-06 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092449+0000", 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8727999999999997e-05 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092468+0000", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8700999999999999e-05 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094004+0000", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001535879 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094049+0000", 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4954999999999999e-05 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1159 3.0 e28/23 3:003f5f29:::rbd_data.11769efc726f.0000000000000017:head v 28'8, mlcod=28'5)", 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.23662513299999999, 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001746729, 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8700000000000004e-07 2024-09-18T01:07:35.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094988+0000", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012142999999999999 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095003+0000", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4887e-05 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095011+0000", 2024-09-18T01:07:35.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3839999999999997e-06 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095063+0000", 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1836000000000003e-05 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095081+0000", 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8447000000000001e-05 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.096570+0000", 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014888049999999999 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.096612+0000", 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2052999999999999e-05 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1454 3.0 e28/23 3:038016b9:::rbd_data.11769efc726f.0000000000000021:head v 28'9, mlcod=28'7)", 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.22307290799999999, 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017690220000000001, 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108419+0000", 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.46e-06 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108533+0000", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011414200000000001 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108554+0000", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0253000000000001e-05 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108564+0000", 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0781000000000001e-05 2024-09-18T01:07:35.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108630+0000", 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5920999999999994e-05 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108651+0000", 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0514999999999999e-05 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.110120+0000", 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001469517 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.110187+0000", 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6433000000000006e-05 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:07:35.381 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.213463661, 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1568 3.9 e28/23 3:92680351:::rbd_data.11769efc726f.000000000000002b:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.20488928200000001, 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026420340000000001, 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126603+0000", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.215e-06 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126978+0000", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037491299999999999 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126997+0000", 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9375e-05 2024-09-18T01:07:35.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127007+0000", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127067+0000", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9812000000000003e-05 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127090+0000", 2024-09-18T01:07:35.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3828e-05 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129205+0000", 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002114215 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129243+0000", 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8751000000000002e-05 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1665 3.9 e28/23 3:94cc305d:::rbd_data.11769efc726f.000000000000002c:head v 28'4, mlcod=28'1)", 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.197885159, 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002086358, 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133607+0000", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2309999999999999e-06 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133795+0000", 2024-09-18T01:07:35.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018814299999999999 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133814+0000", 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9386999999999999e-05 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133824+0000", 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-06 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133881+0000", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6875e-05 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133908+0000", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7008e-05 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.135645+0000", 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001737468 2024-09-18T01:07:35.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.135692+0000", 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6446000000000002e-05 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1732 3.4 e28/23 3:23138acc:::rbd_data.11769efc726f.000000000000002e:head v 28'6, mlcod=26'4)", 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.18804279800000001, 2024-09-18T01:07:35.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001815129, 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:35.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143449+0000", 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.142e-06 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143566+0000", 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116805 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143584+0000", 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7757999999999998e-05 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143593+0000", 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143655+0000", 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1797000000000005e-05 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143679+0000", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3451e-05 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.145218+0000", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015388279999999999 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.145263+0000", 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5426000000000002e-05 2024-09-18T01:07:35.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2196 3.4 e28/23 3:26396459:::rbd_data.11769efc726f.000000000000003e:head v 28'7, mlcod=26'5)", 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.12514489600000001, 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028009300000000001, 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206347+0000", 2024-09-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.341e-06 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206440+0000", 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3080999999999999e-05 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206458+0000", 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7555999999999999e-05 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206467+0000", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1519999999999998e-06 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206543+0000", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5899e-05 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206565+0000", 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2497000000000002e-05 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209098+0000", 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025328080000000001 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209147+0000", 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8596e-05 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.123570445, 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:07:35.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:07:35.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.12197469700000001, 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:07:35.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.12116407799999999, 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:07:35.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:07:35.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2797 3.4 e28/23 3:20519203:::rbd_data.11769efc726f.00000000000000aa:head v 28'12, mlcod=28'10)", 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.077599522000000004, 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001717566, 2024-09-18T01:07:35.418 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253893+0000", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.42e-06 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253960+0000", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6956000000000001e-05 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253973+0000", 2024-09-18T01:07:35.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3827e-05 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253983+0000", 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3430000000000002e-06 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.254043+0000", 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.033e-05 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.254066+0000", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3476e-05 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.255554+0000", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014877720000000001 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.255609+0000", 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4441999999999998e-05 2024-09-18T01:07:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.070616584999999996, 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:07:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.427 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:35.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:35.429 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:35.427+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:35.530 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:35.529+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:35.531 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:35.531 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:35.532 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:35.532 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:35.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:35.631 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:35.630+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:35.643 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:35.643 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:35.644 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:35.644 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:35.644 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:35.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:35.736 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:35.735+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:35.798 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:35.798 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:35.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:35.798 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:35.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1542 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1638400~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.67928777500000004, 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085099709999999999, 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1542 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.799 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116934+0000", 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116936+0000", 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0760000000000001e-06 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116942+0000", 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116946+0000", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125023+0000", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080768239999999998 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125148+0000", 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012473000000000001 2024-09-18T01:07:35.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125443+0000", 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029539300000000002 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1543 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:35.803 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.67926855600000002, 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087055110000000008, 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1543 2024-09-18T01:07:35.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116953+0000", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116954+0000", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7680000000000001e-06 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116961+0000", 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0600000000000002e-06 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116966+0000", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125475+0000", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085082920000000006 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125553+0000", 2024-09-18T01:07:35.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8374999999999994e-05 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125658+0000", 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104751 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1544 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1900544~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.808 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.67924825300000002, 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090197200000000002, 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1544 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116976+0000", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8499999999999998e-06 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116983+0000", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0600000000000002e-06 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116988+0000", 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4319999999999998e-06 2024-09-18T01:07:35.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125723+0000", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087347350000000004 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125849+0000", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126477 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125992+0000", 2024-09-18T01:07:35.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014282800000000001 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1545 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 2031616~7425 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.67922612400000004, 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094544480000000007, 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.813 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1545 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:35.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116997+0000", 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1150000000000001e-06 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117003+0000", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9549999999999997e-06 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117008+0000", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7659999999999998e-06 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126113+0000", 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091051299999999995 2024-09-18T01:07:35.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126194+0000", 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0916999999999998e-05 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126449+0000", 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025517399999999999 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1549 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.67827746700000002, 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091394830000000003, 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1549 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117944+0000", 2024-09-18T01:07:35.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117948+0000", 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1880000000000004e-06 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117965+0000", 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6490999999999998e-05 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117971+0000", 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4300000000000003e-06 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127000+0000", 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090283580000000002 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127039+0000", 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9269e-05 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127083+0000", 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3875000000000003e-05 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1641 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66644115500000001, 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0082203109999999992, 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.822 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1641 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129782+0000", 2024-09-18T01:07:35.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.889e-06 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129788+0000", 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1410000000000002e-06 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129794+0000", 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0419999999999999e-06 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.137914+0000", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081194460000000006 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.137951+0000", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7567999999999997e-05 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138000+0000", 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8884000000000002e-05 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.826 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1642 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66642154600000003, 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0084096650000000002, 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.827 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1642 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129800+0000", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129802+0000", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.559e-06 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129808+0000", 2024-09-18T01:07:35.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6609999999999997e-06 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129812+0000", 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3919999999999996e-06 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138035+0000", 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0082228539999999999 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138099+0000", 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4355000000000003e-05 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138209+0000", 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109485 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1643 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.831 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66640296200000004, 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085459090000000008, 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1643 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129821+0000", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5040000000000001e-06 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129827+0000", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9959999999999999e-06 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129832+0000", 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8029999999999996e-06 2024-09-18T01:07:35.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138246+0000", 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0084141389999999993 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138309+0000", 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3208999999999997e-05 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138364+0000", 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4867999999999997e-05 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1644 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66638362500000003, 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086601939999999995, 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1644 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129838+0000", 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129839+0000", 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129845+0000", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0009999999999998e-06 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129851+0000", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999998e-06 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138388+0000", 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008537088 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138436+0000", 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8439999999999997e-05 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138497+0000", 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1067000000000003e-05 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1645 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66636357499999999, 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087626470000000001, 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1645 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129858+0000", 2024-09-18T01:07:35.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7800000000000002e-07 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129875+0000", 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7810000000000001e-05 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129893+0000", 2024-09-18T01:07:35.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7631000000000001e-05 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129899+0000", 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.502e-06 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138529+0000", 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086300379999999996 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138571+0000", 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2196000000000001e-05 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138620+0000", 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9091999999999997e-05 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1646 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.845 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66631403899999997, 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088320880000000001, 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1646 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:35.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129910+0000", 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129916+0000", 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129920+0000", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.549e-06 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138646+0000", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008725198 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138685+0000", 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9839000000000003e-05 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138739+0000", 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3411000000000001e-05 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1647 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.850 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66629524600000001, 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089334700000000006, 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1647 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1100000000000002e-07 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129928+0000", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7859999999999999e-06 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129934+0000", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129938+0000", 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:07:35.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138766+0000", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088282580000000003 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138807+0000", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0249e-05 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138859+0000", 2024-09-18T01:07:35.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2441000000000002e-05 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1648 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66627704899999995, 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090474479999999996, 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1648 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:35.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.72e-07 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129946+0000", 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.84e-06 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129953+0000", 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6630000000000004e-06 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129957+0000", 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7729999999999999e-06 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138892+0000", 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089348339999999991 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138943+0000", 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0719e-05 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138991+0000", 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8247000000000001e-05 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.859 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1649 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.666257196, 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091342990000000002, 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1649 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:07:35.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129966+0000", 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0200000000000001e-06 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129972+0000", 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129977+0000", 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6940000000000003e-06 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139016+0000", 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090390760000000001 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139052+0000", 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5581999999999999e-05 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139098+0000", 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6372999999999998e-05 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1650 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66623823900000001, 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092311730000000005, 2024-09-18T01:07:35.864 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1650 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.34e-07 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129985+0000", 2024-09-18T01:07:35.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.795e-06 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129990+0000", 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7509999999999997e-06 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129995+0000", 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6879999999999998e-06 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139123+0000", 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.009127352 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139159+0000", 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6854999999999999e-05 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139214+0000", 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4398000000000003e-05 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.868 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1651 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66622044499999999, 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0093252720000000008, 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1651 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130001+0000", 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2300000000000002e-07 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130002+0000", 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.666e-06 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130008+0000", 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130014+0000", 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139239+0000", 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092252210000000005 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139275+0000", 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6315e-05 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139326+0000", 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0297999999999999e-05 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.873 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1652 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66620140800000005, 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094152379999999994, 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1652 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130020+0000", 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130022+0000", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8360000000000001e-06 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130028+0000", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8980000000000001e-06 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130033+0000", 2024-09-18T01:07:35.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7270000000000004e-06 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139351+0000", 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0093175089999999999 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139386+0000", 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5669e-05 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139435+0000", 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8248000000000003e-05 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1653 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.66618146099999997, 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0095145769999999998, 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.878 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1653 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130040+0000", 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130043+0000", 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4350000000000001e-06 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130049+0000", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0599999999999996e-06 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130055+0000", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2889999999999997e-06 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139461+0000", 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094062940000000008 2024-09-18T01:07:35.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139502+0000", 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0689000000000003e-05 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139554+0000", 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2426000000000003e-05 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1654 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.666158272, 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0095921170000000007, 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1654 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:07:35.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130065+0000", 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0509999999999999e-06 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130071+0000", 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130077+0000", 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139571+0000", 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094942619999999998 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139607+0000", 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5902999999999998e-05 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139655+0000", 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7543999999999997e-05 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2335 3.8 3:10eeae9a:::rbd_data.11769efc726f.000000000000004e:head [write 524288~219230 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.59055623300000004, 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089538670000000008, 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2335 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205666+0000", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.612e-06 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205747+0000", 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0556999999999995e-05 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205760+0000", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3380999999999999e-05 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205769+0000", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3869999999999995e-06 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205948+0000", 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017957300000000001 2024-09-18T01:07:35.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206075+0000", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012644899999999999 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206213+0000", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013862600000000001 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207716+0000", 2024-09-18T01:07:35.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001502979 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214514+0000", 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067979319999999996 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214572+0000", 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8387000000000002e-05 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:35.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214618+0000", 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5983999999999999e-05 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:35.893 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:35.894 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:35.895 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:35.893+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:35.995 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:35.994+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:36.003 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:36.003 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:36.003 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:36.003 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:36.004 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:07:36.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:36.015 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:07:36.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:07:36.080 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:36.096 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:36.095+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:36.117 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:36.117 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:36.125 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:36.159 INFO:teuthology.orchestra.run.smithi120.stdout:137 write 0x4293bac thru 0x42a0b78 (0xcfcd bytes) 2024-09-18T01:07:36.173 INFO:teuthology.orchestra.run.smithi120.stdout:138 read 0x36804af thru 0x3689a94 (0x95e6 bytes) 2024-09-18T01:07:36.178 INFO:teuthology.orchestra.run.smithi120.stdout:139 writesame 0x5d51be8 thru 0x5d573f4 data_size 0x1(0x580d bytes) 2024-09-18T01:07:36.192 INFO:teuthology.orchestra.run.smithi120.stdout:140 read 0x55775bb thru 0x557ed3a (0x7780 bytes) 2024-09-18T01:07:36.192 INFO:teuthology.orchestra.run.smithi120.stdout:143 writesame 0xd62e23f thru 0xd634d21 data_size 0x7(0x6ae3 bytes) 2024-09-18T01:07:36.204 INFO:teuthology.orchestra.run.smithi120.stdout:144 trunc from 0xe8e66cd to 0x95be6cf 2024-09-18T01:07:36.209 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:36.208+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:36.246 INFO:teuthology.orchestra.run.smithi120.stdout:146 write 0xd0782a8 thru 0xd07d089 (0x4de2 bytes) 2024-09-18T01:07:36.251 INFO:teuthology.orchestra.run.smithi120.stdout:148 read 0x2da7c5 thru 0x2e6a65 (0xc2a1 bytes) 2024-09-18T01:07:36.251 INFO:teuthology.orchestra.run.smithi120.stdout:149 write 0xcfd3b1c thru 0xcfdd266 (0x974b bytes) 2024-09-18T01:07:36.258 INFO:teuthology.orchestra.run.smithi120.stdout:151 write 0x8e410f thru 0x8efe1f (0xbd11 bytes) 2024-09-18T01:07:36.261 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:36.262 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:36.262 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:36.262 INFO:teuthology.orchestra.run.smithi120.stdout:153 write 0x98ffe67 thru 0x99024cc (0x2666 bytes) 2024-09-18T01:07:36.267 INFO:teuthology.orchestra.run.smithi120.stdout:156 write 0xbb5092f thru 0xbb5d3b4 (0xca86 bytes) 2024-09-18T01:07:36.274 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:36.274 INFO:teuthology.orchestra.run.smithi120.stdout:158 writesame 0x6921311 thru 0x69272ff data_size 0x1(0x5fef bytes) 2024-09-18T01:07:36.283 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:36.283 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:36.283 INFO:teuthology.orchestra.run.smithi120.stdout:159 write 0x8013de2 thru 0x80208ce (0xcaed bytes) 2024-09-18T01:07:36.285 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3092237329999996, 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079564049999999997, 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974232+0000", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974246+0000", 2024-09-18T01:07:36.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4326e-05 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974271+0000", 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.525e-05 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974286+0000", 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4653e-05 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974374+0000", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8067999999999998e-05 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974382+0000", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5870000000000004e-06 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.976986+0000", 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026044570000000001 2024-09-18T01:07:36.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977054+0000", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7533000000000005e-05 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977201+0000", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014775300000000001 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.978421+0000", 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012200780000000001 2024-09-18T01:07:36.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982094+0000", 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036729810000000001 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982145+0000", 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0970999999999997e-05 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982185+0000", 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9878e-05 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:366 3.1a 3:5e4f253b:::rbd_header.11769efc726f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:34.024454+0000", 2024-09-18T01:07:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.2589982040000001, 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040873330000000003, 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 366 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024454+0000", 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024454+0000", 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024455+0000", 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2500000000000004e-07 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024459+0000", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2479999999999998e-06 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024469+0000", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8779999999999993e-06 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024474+0000", 2024-09-18T01:07:36.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9980000000000003e-06 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024511+0000", 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7107999999999998e-05 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024590+0000", 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8560999999999995e-05 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.024720+0000", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012981000000000001 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.025143+0000", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042314300000000001 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.028390+0000", 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032466750000000001 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.028471+0000", 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0984999999999998e-05 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:34.028542+0000", 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1101999999999998e-05 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:667 3.3 3:c5acc749:::rbd_data.11769efc726f.0000000000000001:head [write 0~49152 [fadvise_sequential+fadvise_nocache],write 524288~327680 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.252533286, 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055416620000000002, 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 667 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030921+0000", 2024-09-18T01:07:36.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.759e-06 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031516+0000", 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00059472799999999999 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031533+0000", 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7158000000000001e-05 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031543+0000", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0309000000000001e-05 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031620+0000", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6312999999999999e-05 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031746+0000", 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012654499999999999 2024-09-18T01:07:36.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031912+0000", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000166059 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.034095+0000", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002183361 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036390+0000", 2024-09-18T01:07:36.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022942010000000001 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036426+0000", 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6143999999999998e-05 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036461+0000", 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5085e-05 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1075 3.14 3:2ca96f7b:::rbd_data.11769efc726f.0000000000000012:head [write 0~344064 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.2154725550000001, 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050345770000000001, 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1075 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067982+0000", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.03e-06 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068283+0000", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030097300000000001 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068296+0000", 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2802999999999999e-05 2024-09-18T01:07:36.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068302+0000", 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2400000000000004e-06 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068361+0000", 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9305000000000002e-05 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068505+0000", 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143805 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068663+0000", 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015771000000000001 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.069688+0000", 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010252079999999999 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.072939+0000", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032505440000000002 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.072974+0000", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4919999999999998e-05 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.073015+0000", 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1038999999999998e-05 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1144 3.9 3:95bdb02d:::rbd_data.11769efc726f.0000000000000016:head [write 0~327680 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.090363+0000", 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.1930890700000001, 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039773539999999998, 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.312 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1144 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090363+0000", 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090363+0000", 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090364+0000", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0699999999999996e-07 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090724+0000", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035939499999999998 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090736+0000", 2024-09-18T01:07:36.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1916000000000001e-05 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090744+0000", 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2700000000000004e-06 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090814+0000", 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9529000000000005e-05 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.090924+0000", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110592 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.091016+0000", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2015000000000006e-05 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.092364+0000", 2024-09-18T01:07:36.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013476600000000001 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.094249+0000", 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018852809999999999 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.094299+0000", 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9480000000000001e-05 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.094341+0000", 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2308999999999998e-05 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1568 3.9 3:92680351:::rbd_data.11769efc726f.000000000000002b:head [write 0~180224 [fadvise_sequential+fadvise_nocache],write 524288~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.157773001, 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044787029999999997, 2024-09-18T01:07:36.318 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1568 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125682+0000", 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126115+0000", 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00043310199999999999 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126134+0000", 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9153000000000001e-05 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126147+0000", 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2729e-05 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126207+0000", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0566000000000003e-05 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126309+0000", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102136 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126424+0000", 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114338 2024-09-18T01:07:36.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129989+0000", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035651300000000001 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130057+0000", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8522999999999994e-05 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130112+0000", 2024-09-18T01:07:36.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4463999999999999e-05 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130158+0000", 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6468999999999998e-05 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1574 3.9 3:91e76528:::rbd_data.11769efc726f.0000000000000030:head [write 524288~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.1552745849999999, 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052526070000000003, 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1574 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128179+0000", 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-06 2024-09-18T01:07:36.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128272+0000", 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2972000000000004e-05 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128286+0000", 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3465999999999999e-05 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128294+0000", 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1140000000000006e-06 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128351+0000", 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7328999999999999e-05 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128484+0000", 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000132485 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128624+0000", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140748 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130198+0000", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015739110000000001 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133368+0000", 2024-09-18T01:07:36.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031694330000000001 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133404+0000", 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6724999999999998e-05 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133431+0000", 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6225999999999999e-05 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.330 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1665 3.9 3:94cc305d:::rbd_data.11769efc726f.000000000000002c:head [write 0~180224 [fadvise_sequential+fadvise_nocache],write 524288~253185 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.131999+0000", 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.151453756, 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039596140000000002, 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.331 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1665 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.131999+0000", 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.131999+0000", 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132000+0000", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3399999999999997e-07 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132377+0000", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037749700000000002 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132389+0000", 2024-09-18T01:07:36.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1435e-05 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132394+0000", 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132434+0000", 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0197000000000002e-05 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132516+0000", 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2188999999999995e-05 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.132619+0000", 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103062 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133778+0000", 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001158362 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.135885+0000", 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021078099999999999 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.135932+0000", 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7083999999999998e-05 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.135958+0000", 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5984999999999999e-05 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:2033 3.1c e28/23 3:3a60bf14:::rbd_data.11769efc726f.0000000000000041:head v 28'4, mlcod=27'2)", 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.174215+0000", 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.1092378410000001, 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044013799999999999, 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174215+0000", 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174215+0000", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174216+0000", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2099999999999995e-07 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174320+0000", 2024-09-18T01:07:36.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010474799999999999 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174331+0000", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0355e-05 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174338+0000", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3780000000000003e-06 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174393+0000", 2024-09-18T01:07:36.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5359000000000003e-05 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.174416+0000", 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2033999999999999e-05 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178554+0000", 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041386369999999997 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178616+0000", 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1947999999999997e-05 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2197 3.14 3:2e019870:::rbd_data.11769efc726f.0000000000000042:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.1071284910000001, 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076807569999999999, 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.342 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2197 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176325+0000", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3500000000000005e-07 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176902+0000", 2024-09-18T01:07:36.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057736500000000004 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176915+0000", 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2201000000000001e-05 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176920+0000", 2024-09-18T01:07:36.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.164e-06 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176959+0000", 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8955000000000002e-05 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177051+0000", 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2440000000000003e-05 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177147+0000", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5685000000000003e-05 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181258+0000", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041109440000000001 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183932+0000", 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002674184 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183971+0000", 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8621000000000002e-05 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.184005+0000", 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4263e-05 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:2199 3.1d e28/23 3:bd875124:::rbd_data.11769efc726f.000000000000003f:head v 28'4, mlcod=26'2)", 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.105590834, 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085995269999999992, 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177863+0000", 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177946+0000", 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3165999999999996e-05 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177956+0000", 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0617e-05 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177963+0000", 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5470000000000002e-06 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178020+0000", 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6954999999999999e-05 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178039+0000", 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.965e-05 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186422+0000", 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083826030000000006 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186461+0000", 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9169999999999999e-05 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2217 3.1a 3:591c9277:::rbd_data.11769efc726f.0000000000000048:head [write 524288~229376 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.353 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.1017016900000001, 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049731319999999999, 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2217 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181752+0000", 2024-09-18T01:07:36.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181952+0000", 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020039 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181963+0000", 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0747000000000001e-05 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181968+0000", 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3730000000000001e-06 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182010+0000", 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1737999999999999e-05 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182094+0000", 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3364999999999999e-05 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182177+0000", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3053000000000006e-05 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183908+0000", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017312250000000001 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186667+0000", 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027587470000000002 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186700+0000", 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3642000000000001e-05 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186724+0000", 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3782999999999999e-05 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2273 3.1a 3:5df80eb9:::rbd_data.11769efc726f.0000000000000044:head [write 0~3312 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0914963099999999, 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047637039999999997, 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2273 2024-09-18T01:07:36.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191957+0000", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0299999999999997e-07 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192491+0000", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053392200000000004 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192505+0000", 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4428e-05 2024-09-18T01:07:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192514+0000", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2349999999999999e-06 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192566+0000", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2395000000000002e-05 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192693+0000", 2024-09-18T01:07:36.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000126573 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192845+0000", 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015218799999999999 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.195787+0000", 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029419540000000001 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196621+0000", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083438100000000003 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196679+0000", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7803000000000002e-05 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196720+0000", 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0921999999999999e-05 2024-09-18T01:07:36.365 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2311 3.3 3:c5270ba5:::rbd_data.11769efc726f.0000000000000045:head [write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.197252+0000", 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.086200329, 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039997720000000004, 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2311 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197252+0000", 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197252+0000", 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197253+0000", 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.035e-06 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197663+0000", 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040981699999999998 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197674+0000", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197679+0000", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2639999999999999e-06 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197722+0000", 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3319999999999999e-05 2024-09-18T01:07:36.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197812+0000", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9550999999999997e-05 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.197907+0000", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5610000000000001e-05 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.199401+0000", 2024-09-18T01:07:36.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001493849 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.201174+0000", 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001773249 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.201210+0000", 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5759000000000001e-05 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.201252+0000", 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1832000000000002e-05 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3082 3.1a 3:5e4f253b:::rbd_header.11769efc726f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.27644200299999999, 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044404190000000001, 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.372 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3082 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007012+0000", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.604e-06 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007019+0000", 2024-09-18T01:07:36.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3150000000000003e-06 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007031+0000", 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1676e-05 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007038+0000", 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2790000000000001e-06 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007084+0000", 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007089+0000", 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3800000000000004e-06 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.009937+0000", 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002848363 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.010002+0000", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4672999999999996e-05 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.010363+0000", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036154399999999999 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011226+0000", 2024-09-18T01:07:36.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000862618 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011390+0000", 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016363200000000001 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011423+0000", 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3368999999999998e-05 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011451+0000", 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7948000000000001e-05 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3106 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'15, mlcod=28'13)", 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.161469+0000", 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.121983586, 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043839980000000001, 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161469+0000", 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161469+0000", 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161470+0000", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161475+0000", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.904e-06 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161485+0000", 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9869999999999994e-06 2024-09-18T01:07:36.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161489+0000", 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4839999999999998e-06 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161533+0000", 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4053e-05 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.161548+0000", 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4970999999999999e-05 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.165822+0000", 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042737540000000003 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.165853+0000", 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1029999999999999e-05 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.383 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3111 3.3 3:c5270ba5:::rbd_data.11769efc726f.0000000000000045:head [write 327680~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.116703036, 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064345019999999999, 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3111 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166768+0000", 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8447000000000001e-05 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166774+0000", 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5620000000000003e-06 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166777+0000", 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1599999999999998e-06 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166827+0000", 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0037999999999998e-05 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166926+0000", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8954000000000002e-05 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.167073+0000", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147317 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.169081+0000", 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002007708 2024-09-18T01:07:36.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173084+0000", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040024220000000003 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173139+0000", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5689999999999997e-05 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173184+0000", 2024-09-18T01:07:36.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4821000000000003e-05 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3118 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'16, mlcod=29'14)", 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10417832, 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048919489999999996, 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179275+0000", 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4000000000000001e-07 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179282+0000", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1570000000000003e-06 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179291+0000", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179302+0000", 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0988000000000001e-05 2024-09-18T01:07:36.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179338+0000", 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6052e-05 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179354+0000", 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5452e-05 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184107+0000", 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047530009999999998 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184166+0000", 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9366e-05 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3119 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'17, mlcod=29'14)", 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103357635, 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044026270000000001, 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5000000000000003e-07 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180100+0000", 2024-09-18T01:07:36.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9080000000000003e-06 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180109+0000", 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1610000000000003e-06 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180113+0000", 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7350000000000004e-06 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180153+0000", 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9233999999999997e-05 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180165+0000", 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2682e-05 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184457+0000", 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042922389999999998 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184498+0000", 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0117999999999998e-05 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3164 3.1a 3:59c1a3f5:::rbd_data.11769efc726f.0000000000000097:head [truncate 747215] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.062915158999999998, 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045821480000000003, 2024-09-18T01:07:36.399 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3164 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220539+0000", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.443e-06 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220546+0000", 2024-09-18T01:07:36.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1509999999999998e-06 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220561+0000", 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4555999999999999e-05 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220568+0000", 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6040000000000004e-06 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220626+0000", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7821000000000001e-05 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220711+0000", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5239000000000002e-05 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220831+0000", 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011966100000000001 2024-09-18T01:07:36.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.221532+0000", 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000701288 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225056+0000", 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035236970000000001 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:36.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225091+0000", 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5540999999999997e-05 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225120+0000", 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8147e-05 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:36.405 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:160 trunc from 0xd07d08a to 0xaa27625 2024-09-18T01:07:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:36.407 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:36.408 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:161 write 0xb2a2268 thru 0xb2aeae5 (0xc87e bytes) 2024-09-18T01:07:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xe15052f 2024-09-18T01:07:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:162 trunc from 0xb2aeae6 to 0xe15052f 2024-09-18T01:07:36.409 INFO:teuthology.orchestra.run.smithi120.stdout:165 writesame 0x3ffc531 thru 0x400a459 data_size 0x3(0xdf29 bytes) 2024-09-18T01:07:36.409 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:36.409 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:166 write 0x74e0895 thru 0x74e1924 (0x1090 bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:167 writesame 0x9fcd7b3 thru 0x9fd6d78 data_size 0x13(0x95c6 bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:168 write 0x72ec21a thru 0x72f4580 (0x8367 bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:169 punch from 0x769aee to 0x76fd46, (0x6258 bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:170 read 0x2bc74bf thru 0x2bd2de5 (0xb927 bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:172 write 0x4ef1462 thru 0x4f0017c (0xed1b bytes) 2024-09-18T01:07:36.410 INFO:teuthology.orchestra.run.smithi120.stdout:173 read 0x72bd3f2 thru 0x72cd2cc (0xfedb bytes) 2024-09-18T01:07:36.411 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:36.411 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:36.411 INFO:teuthology.orchestra.run.smithi120.stdout:174 trunc from 0xe15052f to 0x6438775 2024-09-18T01:07:36.412 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:36.406+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:36.412 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:36.412 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:36.417 INFO:teuthology.orchestra.run.smithi120.stdout:175 read 0x256080f thru 0x2561bb8 (0x13aa bytes) 2024-09-18T01:07:36.417 INFO:teuthology.orchestra.run.smithi120.stdout:176 read 0xa16281 thru 0xa260c4 (0xfe44 bytes) 2024-09-18T01:07:36.431 INFO:teuthology.orchestra.run.smithi120.stdout:179 write 0xce6d5ac thru 0xce70471 (0x2ec6 bytes) 2024-09-18T01:07:36.438 INFO:teuthology.orchestra.run.smithi120.stdout:182 write 0xce8e353 thru 0xce8e5d3 (0x281 bytes) 2024-09-18T01:07:36.440 INFO:teuthology.orchestra.run.smithi120.stdout:183 write 0x88d9df2 thru 0x88e494d (0xab5c bytes) 2024-09-18T01:07:36.441 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:36.441 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:36.456 INFO:teuthology.orchestra.run.smithi120.stdout:184 write 0x70c4c47 thru 0x70d3a77 (0xee31 bytes) 2024-09-18T01:07:36.459 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:36.467 INFO:teuthology.orchestra.run.smithi120.stdout:188 punch from 0x6c8a3dc to 0x6c95530, (0xb154 bytes) 2024-09-18T01:07:36.467 INFO:teuthology.orchestra.run.smithi120.stdout:189 write 0x50b6b3e thru 0x50bb764 (0x4c27 bytes) 2024-09-18T01:07:36.471 INFO:teuthology.orchestra.run.smithi120.stdout:190 writesame 0x7398a94 thru 0x739da15 data_size 0x1(0x4f82 bytes) 2024-09-18T01:07:36.479 INFO:teuthology.orchestra.run.smithi120.stdout:194 write 0xb75d483 thru 0xb761a61 (0x45df bytes) 2024-09-18T01:07:36.483 INFO:teuthology.orchestra.run.smithi120.stdout:196 writesame 0x4f6029 thru 0x501055 data_size 0x11(0xb02d bytes) 2024-09-18T01:07:36.488 INFO:teuthology.orchestra.run.smithi120.stdout:197 read 0xfec73a thru 0xff3e82 (0x7749 bytes) 2024-09-18T01:07:36.489 INFO:teuthology.orchestra.run.smithi120.stdout:198 punch from 0x68ec0d3 to 0x68f859a, (0xc4c7 bytes) 2024-09-18T01:07:36.494 INFO:teuthology.orchestra.run.smithi120.stdout:199 write 0xd3e2ea1 thru 0xd3ef2d0 (0xc430 bytes) 2024-09-18T01:07:36.500 INFO:teuthology.orchestra.run.smithi120.stdout:200 read 0x67f5704 thru 0x6805342 (0xfc3f bytes) 2024-09-18T01:07:36.503 INFO:teuthology.orchestra.run.smithi120.stdout:201 trunc from 0xd3ef2d1 to 0xb4aec7f 2024-09-18T01:07:36.508 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:36.507+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:36.518 INFO:teuthology.orchestra.run.smithi120.stdout:202 writesame 0x9cad8de thru 0x9cb415f data_size 0x1a(0x6882 bytes) 2024-09-18T01:07:36.523 INFO:teuthology.orchestra.run.smithi120.stdout:203 write 0x6bfecbd thru 0x6c053ff (0x6743 bytes) 2024-09-18T01:07:36.527 INFO:teuthology.orchestra.run.smithi120.stdout:206 writesame 0x4874640 thru 0x4880291 data_size 0xf(0xbc52 bytes) 2024-09-18T01:07:36.531 INFO:teuthology.orchestra.run.smithi120.stdout:207 write 0xb425061 thru 0xb431d3d (0xccdd bytes) 2024-09-18T01:07:36.537 INFO:teuthology.orchestra.run.smithi120.stdout:208 punch from 0x1ccd118 to 0x1cdc656, (0xf53e bytes) 2024-09-18T01:07:36.537 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:36.542 INFO:teuthology.orchestra.run.smithi120.stdout:213 trunc from 0xb4aec7f to 0x6fb7b36 2024-09-18T01:07:36.563 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:36.563 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:36.568 INFO:teuthology.orchestra.run.smithi120.stdout:214 punch from 0x1fdddb7 to 0x1fea7d8, (0xca21 bytes) 2024-09-18T01:07:36.568 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:36.569 INFO:teuthology.orchestra.run.smithi120.stdout:216 punch from 0x25f20bb to 0x25fab51, (0x8a96 bytes) 2024-09-18T01:07:36.572 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:36.578 INFO:teuthology.orchestra.run.smithi120.stdout:220 writesame 0x9521d74 thru 0x952a9dd data_size 0x9(0x8c6a bytes) 2024-09-18T01:07:36.578 INFO:teuthology.orchestra.run.smithi120.stdout:221 write 0x246fa36 thru 0x246ffdb (0x5a6 bytes) 2024-09-18T01:07:36.579 INFO:teuthology.orchestra.run.smithi120.stdout:222 read 0x3039927 thru 0x303abc5 (0x129f bytes) 2024-09-18T01:07:36.584 INFO:teuthology.orchestra.run.smithi120.stdout:223 trunc from 0x952a9de to 0x5bbbac7 2024-09-18T01:07:36.606 INFO:teuthology.orchestra.run.smithi120.stdout:224 read 0x45b91ea thru 0x45c03f1 (0x7208 bytes) 2024-09-18T01:07:36.606 INFO:teuthology.orchestra.run.smithi120.stdout:226 clone 2 order 21 su 262144 sc 7 2024-09-18T01:07:36.608 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:36.607+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:36.710 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:36.709+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:36.715 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:36.715 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:36.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:36.811 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:36.810+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:36.827 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:36.827 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:36.836 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:36.912 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:36.911+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:36.940 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:36.940 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:36.949 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:37.013 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:07:37.014 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:37.012+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:37.019 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:07:37.029 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:07:37.099 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.099 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.107 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:37.117 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:37.116+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:37.211 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.211 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.215 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:37.214+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:37.220 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:37.316 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:37.315+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:37.364 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.364 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.372 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:37.456 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:37.455+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:37.516 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.517 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.518 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:37.517+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:37.525 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:37.618 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:37.618+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:37.669 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.669 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.679 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:37.723 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:37.722+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:37.784 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.784 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:37.834 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:37.833+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:37.923 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:37.922+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:37.935 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:37.935 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:37.944 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:38.018 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:07:38.023 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:38.022+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:38.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:07:38.124 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:38.123+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:38.143 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:38.143 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:38.154 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:38.225 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:38.224+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:38.315 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:38.315 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:38.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:38.325 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:38.324+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:38.427 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:38.426+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:38.469 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:38.469 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:38.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:38.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:38.562 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:38.561+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:38.595 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:38.595 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:38.613 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:38.626 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.446371638, 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:07:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4463528339999998, 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:07:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4463362659999999, 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:07:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:07:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:07:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:07:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:07:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.44632008, 2024-09-18T01:07:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:07:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:07:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:07:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:07:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:07:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:07:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:07:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4463085109999998, 2024-09-18T01:07:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:07:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:07:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:07:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:07:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462991450000002, 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462869619999998, 2024-09-18T01:07:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:07:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:07:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462712670000002, 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:07:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462541130000002, 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:07:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462319470000002, 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:07:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:07:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:07:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462154969999998, 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:07:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:07:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:07:38.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4462060430000001, 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:07:38.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.446171992, 2024-09-18T01:07:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:07:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:07:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4461522680000001, 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:07:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:07:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:07:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4461340310000002, 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:07:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.446113456, 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:07:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:07:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4460942289999998, 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:07:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:07:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.446074876, 2024-09-18T01:07:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:07:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:07:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4460562210000001, 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:07:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:07:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4372742719999998, 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:07:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:07:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:07:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:07:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:38.771 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:38.772 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:38.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:38.775 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:38.775 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:38.776 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:38.776 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:38.777 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:38.777 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:38.777 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:38.778 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:38.778 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:38.779 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:38.771+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:38.873 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:38.872+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:38.917 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:38.917 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:38.918 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:38.918 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:38.925 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:38.974 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:38.973+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:39.029 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:07:39.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:07:39.035 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:39.035 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:39.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:07:39.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:39.075 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:39.074+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:39.117 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-18T01:07:39.176 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:39.175+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:39.212 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.271123051, 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:07:39.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:07:39.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:783 3.0 e28/23 3:04cd6778:::rbd_data.11769efc726f.0000000000000005:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1649319250000003, 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026120100000000001, 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045917+0000", 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0500000000000003e-07 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046106+0000", 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018893600000000001 2024-09-18T01:07:39.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046125+0000", 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9128e-05 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046137+0000", 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046199+0000", 2024-09-18T01:07:39.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2520000000000004e-05 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046222+0000", 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2554000000000001e-05 2024-09-18T01:07:39.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048480+0000", 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002258047 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048528+0000", 2024-09-18T01:07:39.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8241000000000001e-05 2024-09-18T01:07:39.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:870 3.0 e28/23 3:001618d7:::rbd_data.11769efc726f.000000000000000b:head v 28'3, mlcod=27'1)", 2024-09-18T01:07:39.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1511831319999999, 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019550090000000002, 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059665+0000", 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059666+0000", 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.601e-06 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.059981+0000", 2024-09-18T01:07:39.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031502199999999998 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060002+0000", 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0237e-05 2024-09-18T01:07:39.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060011+0000", 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3999999999999998e-06 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060085+0000", 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3720999999999994e-05 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.060112+0000", 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7019e-05 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061577+0000", 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014653559999999999 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061620+0000", 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2653e-05 2024-09-18T01:07:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:871 3.0 e28/23 3:0320ffd9:::rbd_data.11769efc726f.0000000000000011:head v 28'4, mlcod=27'1)", 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1497612459999997, 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00170341, 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061087+0000", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061088+0000", 2024-09-18T01:07:39.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-06 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061674+0000", 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00058667599999999995 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061688+0000", 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3709999999999999e-05 2024-09-18T01:07:39.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061696+0000", 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061761+0000", 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5508999999999999e-05 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.061785+0000", 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3407000000000001e-05 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.062748+0000", 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00096303600000000001 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.062790+0000", 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2348000000000002e-05 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1431950110000004, 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:07:39.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:07:39.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1145 3.0 e28/23 3:0368a151:::rbd_data.11769efc726f.000000000000001b:head v 28'7, mlcod=28'5)", 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1193589800000003, 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026055610000000002, 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:39.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091490+0000", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091544+0000", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3934999999999998e-05 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091563+0000", 2024-09-18T01:07:39.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8423e-05 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091576+0000", 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3010000000000001e-05 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091636+0000", 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0732000000000001e-05 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091658+0000", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1943999999999999e-05 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094060+0000", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00240136 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094095+0000", 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.252 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1144 3.9 e28/23 3:95bdb02d:::rbd_data.11769efc726f.0000000000000016:head v 28'1, mlcod=0'0)", 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1189193460000002, 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021202579999999999, 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091929+0000", 2024-09-18T01:07:39.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092370+0000", 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044077999999999999 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092382+0000", 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2649999999999999e-05 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092391+0000", 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2090000000000005e-06 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092449+0000", 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8727999999999997e-05 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.092468+0000", 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8700999999999999e-05 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094004+0000", 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001535879 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094049+0000", 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4954999999999999e-05 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1159 3.0 e28/23 3:003f5f29:::rbd_data.11769efc726f.0000000000000017:head v 28'8, mlcod=28'5)", 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1159824230000002, 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001746729, 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094866+0000", 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8700000000000004e-07 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094988+0000", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012142999999999999 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095003+0000", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4887e-05 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095011+0000", 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3839999999999997e-06 2024-09-18T01:07:39.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095063+0000", 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1836000000000003e-05 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.095081+0000", 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8447000000000001e-05 2024-09-18T01:07:39.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.096570+0000", 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014888049999999999 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.096612+0000", 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2052999999999999e-05 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1454 3.0 e28/23 3:038016b9:::rbd_data.11769efc726f.0000000000000021:head v 28'9, mlcod=28'7)", 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1024301980000004, 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017690220000000001, 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108418+0000", 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108419+0000", 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.46e-06 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108533+0000", 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011414200000000001 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108554+0000", 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0253000000000001e-05 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108564+0000", 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0781000000000001e-05 2024-09-18T01:07:39.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108630+0000", 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5920999999999994e-05 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.108651+0000", 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0514999999999999e-05 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.110120+0000", 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001469517 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.110187+0000", 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6433000000000006e-05 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.268 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0928209510000002, 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:07:39.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1568 3.9 e28/23 3:92680351:::rbd_data.11769efc726f.000000000000002b:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0842465719999996, 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026420340000000001, 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126603+0000", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.215e-06 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126978+0000", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037491299999999999 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126997+0000", 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9375e-05 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127007+0000", 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127067+0000", 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9812000000000003e-05 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127090+0000", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3828e-05 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129205+0000", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002114215 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129243+0000", 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8751000000000002e-05 2024-09-18T01:07:39.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1665 3.9 e28/23 3:94cc305d:::rbd_data.11769efc726f.000000000000002c:head v 28'4, mlcod=28'1)", 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0772424489999999, 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002086358, 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133605+0000", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133607+0000", 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2309999999999999e-06 2024-09-18T01:07:39.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133795+0000", 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018814299999999999 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133814+0000", 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9386999999999999e-05 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133824+0000", 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-06 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133881+0000", 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6875e-05 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.133908+0000", 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7008e-05 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.135645+0000", 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001737468 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.135692+0000", 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6446000000000002e-05 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.282 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1732 3.4 e28/23 3:23138acc:::rbd_data.11769efc726f.000000000000002e:head v 28'6, mlcod=26'4)", 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0674000880000003, 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001815129, 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143448+0000", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143449+0000", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.142e-06 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143566+0000", 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116805 2024-09-18T01:07:39.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143584+0000", 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7757999999999998e-05 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143593+0000", 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143655+0000", 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1797000000000005e-05 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.143679+0000", 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3451e-05 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.145218+0000", 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015388279999999999 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.145263+0000", 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5426000000000002e-05 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2196 3.4 e28/23 3:26396459:::rbd_data.11769efc726f.000000000000003e:head v 28'7, mlcod=26'5)", 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0045021859999999, 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028009300000000001, 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206347+0000", 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.341e-06 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206440+0000", 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3080999999999999e-05 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206458+0000", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7555999999999999e-05 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206467+0000", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1519999999999998e-06 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206543+0000", 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5899e-05 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206565+0000", 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2497000000000002e-05 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209098+0000", 2024-09-18T01:07:39.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025328080000000001 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209147+0000", 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8596e-05 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:07:39.292 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0029277350000001, 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:07:39.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:07:39.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0013319870000004, 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:07:39.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0005213680000002, 2024-09-18T01:07:39.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:39.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:07:39.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2797 3.4 e28/23 3:20519203:::rbd_data.11769efc726f.00000000000000aa:head v 28'12, mlcod=28'10)", 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.956956812, 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001717566, 2024-09-18T01:07:39.307 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253891+0000", 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253893+0000", 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.42e-06 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253960+0000", 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6956000000000001e-05 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253973+0000", 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3827e-05 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.253983+0000", 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3430000000000002e-06 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.254043+0000", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.033e-05 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.254066+0000", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3476e-05 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.255554+0000", 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014877720000000001 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.255609+0000", 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4441999999999998e-05 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.949973875, 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:07:39.312 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:07:39.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:07:39.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:3134 3.4 e29/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 29'16, mlcod=28'13)", 2024-09-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.0025999400000001, 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031975599999999999, 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208255+0000", 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.319e-06 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208262+0000", 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4309999999999997e-06 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208267+0000", 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0370000000000002e-06 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208575+0000", 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030814599999999997 2024-09-18T01:07:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208594+0000", 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8635999999999999e-05 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211406+0000", 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028115670000000001 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211446+0000", 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0021999999999997e-05 2024-09-18T01:07:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.325 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:39.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:39.328 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:39.326+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:39.400 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:39.400 INFO:teuthology.orchestra.run.smithi120.stdout:229 read 0x33e6fe1 thru 0x33f0994 (0x99b4 bytes) 2024-09-18T01:07:39.401 INFO:teuthology.orchestra.run.smithi120.stdout:230 write 0x919a1f thru 0x924602 (0xabe4 bytes) 2024-09-18T01:07:39.407 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:39.408 INFO:teuthology.orchestra.run.smithi120.stdout:233 read 0xf57c9 thru 0xffcd4 (0xa50c bytes) 2024-09-18T01:07:39.408 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:39.408 INFO:teuthology.orchestra.run.smithi120.stdout:235 read 0xd80f9a thru 0xd8fb45 (0xebac bytes) 2024-09-18T01:07:39.424 INFO:teuthology.orchestra.run.smithi120.stdout:237 write 0xd783000 thru 0xd78ce11 (0x9e12 bytes) 2024-09-18T01:07:39.427 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:39.426+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:39.428 INFO:teuthology.orchestra.run.smithi120.stdout:238 write 0x6ff43c9 thru 0x7001969 (0xd5a1 bytes) 2024-09-18T01:07:39.432 INFO:teuthology.orchestra.run.smithi120.stdout:239 writesame 0x74fa44 thru 0x751e79 data_size 0x12(0x2436 bytes) 2024-09-18T01:07:39.434 INFO:teuthology.orchestra.run.smithi120.stdout:240 read 0x6be564d thru 0x6beebcf (0x9583 bytes) 2024-09-18T01:07:39.434 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:39.434 INFO:teuthology.orchestra.run.smithi120.stdout:244 read 0x16817a6 thru 0x168f31d (0xdb78 bytes) 2024-09-18T01:07:39.434 INFO:teuthology.orchestra.run.smithi120.stdout:245 punch from 0x6c92d7a to 0x6ca1673, (0xe8f9 bytes) 2024-09-18T01:07:39.434 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xe322309 2024-09-18T01:07:39.435 INFO:teuthology.orchestra.run.smithi120.stdout:246 trunc from 0xd78ce12 to 0xe322309 2024-09-18T01:07:39.440 INFO:teuthology.orchestra.run.smithi120.stdout:247 read 0x71bda99 thru 0x71c9eab (0xc413 bytes) 2024-09-18T01:07:39.446 INFO:teuthology.orchestra.run.smithi120.stdout:248 trunc from 0xe322309 to 0x9f476bc 2024-09-18T01:07:39.456 INFO:teuthology.orchestra.run.smithi120.stdout:249 read 0x2d0533c thru 0x2d11387 (0xc04c bytes) 2024-09-18T01:07:39.468 INFO:teuthology.orchestra.run.smithi120.stdout:250 write 0xe4756f2 thru 0xe47afd4 (0x58e3 bytes) 2024-09-18T01:07:39.471 INFO:teuthology.orchestra.run.smithi120.stdout:251 read 0x78ed6f2 thru 0x78fa4e6 (0xcdf5 bytes) 2024-09-18T01:07:39.471 INFO:teuthology.orchestra.run.smithi120.stdout:252 read 0x1424cc3 thru 0x14250f5 (0x433 bytes) 2024-09-18T01:07:39.471 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:39.471 INFO:teuthology.orchestra.run.smithi120.stdout:254 clone 3 order 25 su 524288 sc 15 2024-09-18T01:07:39.480 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:39.481 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:39.481 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:39.481 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:39.490 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:39.529 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:39.528+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:39.532 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:07:39.539 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:07:39.549 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:07:39.639 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:39.637+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:39.642 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:39.642 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:39.651 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:39.735 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:39.733+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:39.799 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:39.799 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:39.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1542 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1638400~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6804933110000002, 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085099709999999999, 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1542 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116933+0000", 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116934+0000", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116936+0000", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0760000000000001e-06 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116942+0000", 2024-09-18T01:07:39.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116946+0000", 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125023+0000", 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080768239999999998 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125148+0000", 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012473000000000001 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125443+0000", 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029539300000000002 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1543 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6804740919999999, 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087055110000000008, 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1543 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116952+0000", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116953+0000", 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116954+0000", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7680000000000001e-06 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116961+0000", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0600000000000002e-06 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116966+0000", 2024-09-18T01:07:39.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125475+0000", 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085082920000000006 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125553+0000", 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8374999999999994e-05 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125658+0000", 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104751 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1544 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1900544~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:39.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6804537889999995, 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090197200000000002, 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1544 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:39.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116973+0000", 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116976+0000", 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8499999999999998e-06 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116983+0000", 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0600000000000002e-06 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116988+0000", 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4319999999999998e-06 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125723+0000", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087347350000000004 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125849+0000", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126477 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.125992+0000", 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014282800000000001 2024-09-18T01:07:39.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1545 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 2031616~7425 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.68043166, 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094544480000000007, 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1545 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:39.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116995+0000", 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.116997+0000", 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1150000000000001e-06 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117003+0000", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9549999999999997e-06 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117008+0000", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7659999999999998e-06 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126113+0000", 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091051299999999995 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126194+0000", 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0916999999999998e-05 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126449+0000", 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025517399999999999 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.818 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1549 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6794830029999996, 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091394830000000003, 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1549 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117943+0000", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117944+0000", 2024-09-18T01:07:39.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117948+0000", 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1880000000000004e-06 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117965+0000", 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6490999999999998e-05 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.117971+0000", 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4300000000000003e-06 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127000+0000", 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090283580000000002 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127039+0000", 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9269e-05 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127083+0000", 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3875000000000003e-05 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1641 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6676466909999998, 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0082203109999999992, 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.823 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1641 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129780+0000", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129782+0000", 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.889e-06 2024-09-18T01:07:39.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129788+0000", 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1410000000000002e-06 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129794+0000", 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0419999999999999e-06 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.137914+0000", 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081194460000000006 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.137951+0000", 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7567999999999997e-05 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138000+0000", 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8884000000000002e-05 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.827 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1642 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6676270820000001, 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0084096650000000002, 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.828 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1642 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129799+0000", 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129800+0000", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129802+0000", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.559e-06 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129808+0000", 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6609999999999997e-06 2024-09-18T01:07:39.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129812+0000", 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3919999999999996e-06 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138035+0000", 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0082228539999999999 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138099+0000", 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4355000000000003e-05 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138209+0000", 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109485 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1643 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6676084979999999, 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085459090000000008, 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1643 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129818+0000", 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129821+0000", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5040000000000001e-06 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129827+0000", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9959999999999999e-06 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129832+0000", 2024-09-18T01:07:39.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8029999999999996e-06 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138246+0000", 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0084141389999999993 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138309+0000", 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3208999999999997e-05 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138364+0000", 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4867999999999997e-05 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1644 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6675891610000004, 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086601939999999995, 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.837 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1644 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129837+0000", 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129838+0000", 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129839+0000", 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129845+0000", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0009999999999998e-06 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129851+0000", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999998e-06 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138388+0000", 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008537088 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138436+0000", 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8439999999999997e-05 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138497+0000", 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1067000000000003e-05 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.841 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1645 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6675691109999997, 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087626470000000001, 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1645 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129857+0000", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129858+0000", 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7800000000000002e-07 2024-09-18T01:07:39.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129875+0000", 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7810000000000001e-05 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129893+0000", 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7631000000000001e-05 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129899+0000", 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.502e-06 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138529+0000", 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086300379999999996 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138571+0000", 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2196000000000001e-05 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138620+0000", 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9091999999999997e-05 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1646 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.667519575, 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088320880000000001, 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1646 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129907+0000", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129910+0000", 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:07:39.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129916+0000", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129920+0000", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.549e-06 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138646+0000", 2024-09-18T01:07:39.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008725198 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138685+0000", 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9839000000000003e-05 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138739+0000", 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3411000000000001e-05 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1647 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6675007820000003, 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089334700000000006, 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.851 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1647 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129926+0000", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1100000000000002e-07 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129928+0000", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7859999999999999e-06 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129934+0000", 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:07:39.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129938+0000", 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138766+0000", 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088282580000000003 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138807+0000", 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0249e-05 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138859+0000", 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2441000000000002e-05 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1648 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.855 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6674825850000001, 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090474479999999996, 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1648 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129944+0000", 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.72e-07 2024-09-18T01:07:39.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129946+0000", 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.84e-06 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129953+0000", 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6630000000000004e-06 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129957+0000", 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7729999999999999e-06 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138892+0000", 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089348339999999991 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138943+0000", 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0719e-05 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.138991+0000", 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8247000000000001e-05 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1649 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6674627319999997, 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091342990000000002, 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.860 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1649 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129964+0000", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129966+0000", 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0200000000000001e-06 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129972+0000", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129977+0000", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6940000000000003e-06 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139016+0000", 2024-09-18T01:07:39.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090390760000000001 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139052+0000", 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5581999999999999e-05 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139098+0000", 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6372999999999998e-05 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.864 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1650 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6674437749999997, 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092311730000000005, 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1650 2024-09-18T01:07:39.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129983+0000", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.34e-07 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129985+0000", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.795e-06 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129990+0000", 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7509999999999997e-06 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129995+0000", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6879999999999998e-06 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139123+0000", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.009127352 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139159+0000", 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6854999999999999e-05 2024-09-18T01:07:39.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139214+0000", 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4398000000000003e-05 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1651 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6674259810000001, 2024-09-18T01:07:39.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0093252720000000008, 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1651 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130000+0000", 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130001+0000", 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2300000000000002e-07 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130002+0000", 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.666e-06 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130008+0000", 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130014+0000", 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139239+0000", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092252210000000005 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139275+0000", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6315e-05 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139326+0000", 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0297999999999999e-05 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.873 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1652 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6674069439999997, 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094152379999999994, 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.874 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1652 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130019+0000", 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130020+0000", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130022+0000", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8360000000000001e-06 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130028+0000", 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8980000000000001e-06 2024-09-18T01:07:39.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130033+0000", 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7270000000000004e-06 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139351+0000", 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0093175089999999999 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139386+0000", 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5669e-05 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139435+0000", 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8248000000000003e-05 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1653 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:39.878 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6673869970000004, 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0095145769999999998, 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1653 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:39.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130039+0000", 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130040+0000", 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130043+0000", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4350000000000001e-06 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130049+0000", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0599999999999996e-06 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130055+0000", 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2889999999999997e-06 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139461+0000", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094062940000000008 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139502+0000", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0689000000000003e-05 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139554+0000", 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2426000000000003e-05 2024-09-18T01:07:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1654 3.b 3:d5afed29:::rbd_data.1176f5693ee0.000000000000000b:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6673638080000002, 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0095921170000000007, 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.883 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1654 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130063+0000", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130065+0000", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0509999999999999e-06 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130071+0000", 2024-09-18T01:07:39.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.130077+0000", 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139571+0000", 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0094942619999999998 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139607+0000", 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5902999999999998e-05 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.139655+0000", 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7543999999999997e-05 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2335 3.8 3:10eeae9a:::rbd_data.11769efc726f.000000000000004e:head [write 524288~219230 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:39.887 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.5917617689999997, 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089538670000000008, 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2335 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205665+0000", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205666+0000", 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.612e-06 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205747+0000", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0556999999999995e-05 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205760+0000", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3380999999999999e-05 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205769+0000", 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3869999999999995e-06 2024-09-18T01:07:39.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205948+0000", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017957300000000001 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206075+0000", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012644899999999999 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206213+0000", 2024-09-18T01:07:39.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013862600000000001 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207716+0000", 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001502979 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214514+0000", 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067979319999999996 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214572+0000", 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8387000000000002e-05 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214618+0000", 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5983999999999999e-05 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:39.893 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:39.895 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:39.895 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:39.893+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:39.996 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:39.995+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:40.002 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:40.003 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.011 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:40.097 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:40.095+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:40.153 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.154 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:40.154 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:40.154 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:40.154 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.162 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:40.197 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:40.196+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:07:40.266 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2897427439999998, 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079564049999999997, 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974232+0000", 2024-09-18T01:07:40.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974246+0000", 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4326e-05 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974271+0000", 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.525e-05 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974286+0000", 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4653e-05 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974374+0000", 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8067999999999998e-05 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974382+0000", 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5870000000000004e-06 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.976986+0000", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026044570000000001 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977054+0000", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7533000000000005e-05 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977201+0000", 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014775300000000001 2024-09-18T01:07:40.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.978421+0000", 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012200780000000001 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982094+0000", 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036729810000000001 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982145+0000", 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0970999999999997e-05 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982185+0000", 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9878e-05 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:667 3.3 3:c5acc749:::rbd_data.11769efc726f.0000000000000001:head [write 0~49152 [fadvise_sequential+fadvise_nocache],write 524288~327680 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2330522970000004, 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055416620000000002, 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 667 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030919+0000", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.030921+0000", 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.759e-06 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031516+0000", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00059472799999999999 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031533+0000", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7158000000000001e-05 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031543+0000", 2024-09-18T01:07:40.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0309000000000001e-05 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031620+0000", 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6312999999999999e-05 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031746+0000", 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012654499999999999 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:40.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.031912+0000", 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000166059 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.034095+0000", 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002183361 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036390+0000", 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022942010000000001 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036426+0000", 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6143999999999998e-05 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.036461+0000", 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5085e-05 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1075 3.14 3:2ca96f7b:::rbd_data.11769efc726f.0000000000000012:head [write 0~344064 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.195991566, 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050345770000000001, 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1075 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067980+0000", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.067982+0000", 2024-09-18T01:07:40.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.03e-06 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068283+0000", 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030097300000000001 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068296+0000", 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2802999999999999e-05 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068302+0000", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2400000000000004e-06 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068361+0000", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9305000000000002e-05 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068505+0000", 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143805 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.068663+0000", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015771000000000001 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.069688+0000", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010252079999999999 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.072939+0000", 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032505440000000002 2024-09-18T01:07:40.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.072974+0000", 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4919999999999998e-05 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.073015+0000", 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1038999999999998e-05 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1568 3.9 3:92680351:::rbd_data.11769efc726f.000000000000002b:head [write 0~180224 [fadvise_sequential+fadvise_nocache],write 524288~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.138292012, 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044787029999999997, 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1568 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125680+0000", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.125682+0000", 2024-09-18T01:07:40.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126115+0000", 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00043310199999999999 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126134+0000", 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9153000000000001e-05 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126147+0000", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2729e-05 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126207+0000", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0566000000000003e-05 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126309+0000", 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102136 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.126424+0000", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114338 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129989+0000", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035651300000000001 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130057+0000", 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8522999999999994e-05 2024-09-18T01:07:40.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130112+0000", 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4463999999999999e-05 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130158+0000", 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6468999999999998e-05 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1574 3.9 3:91e76528:::rbd_data.11769efc726f.0000000000000030:head [write 524288~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1357935960000001, 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052526070000000003, 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1574 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128178+0000", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128179+0000", 2024-09-18T01:07:40.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-06 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128272+0000", 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2972000000000004e-05 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128286+0000", 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3465999999999999e-05 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128294+0000", 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1140000000000006e-06 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128351+0000", 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7328999999999999e-05 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128484+0000", 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000132485 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.128624+0000", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140748 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.130198+0000", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015739110000000001 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133368+0000", 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031694330000000001 2024-09-18T01:07:40.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133404+0000", 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6724999999999998e-05 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.133431+0000", 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6225999999999999e-05 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2197 3.14 3:2e019870:::rbd_data.11769efc726f.0000000000000042:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.0876475020000003, 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076807569999999999, 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2197 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176325+0000", 2024-09-18T01:07:40.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3500000000000005e-07 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176902+0000", 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057736500000000004 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176915+0000", 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2201000000000001e-05 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176920+0000", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.164e-06 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176959+0000", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8955000000000002e-05 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177051+0000", 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2440000000000003e-05 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177147+0000", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5685000000000003e-05 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181258+0000", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041109440000000001 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183932+0000", 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002674184 2024-09-18T01:07:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183971+0000", 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8621000000000002e-05 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.184005+0000", 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4263e-05 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:2199 3.1d e28/23 3:bd875124:::rbd_data.11769efc726f.000000000000003f:head v 28'4, mlcod=26'2)", 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.0861098450000002, 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085995269999999992, 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177863+0000", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177946+0000", 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3165999999999996e-05 2024-09-18T01:07:40.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177956+0000", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0617e-05 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177963+0000", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5470000000000002e-06 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178020+0000", 2024-09-18T01:07:40.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6954999999999999e-05 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178039+0000", 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.965e-05 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186422+0000", 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083826030000000006 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186461+0000", 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9169999999999999e-05 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2217 3.1a 3:591c9277:::rbd_data.11769efc726f.0000000000000048:head [write 524288~229376 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.0822207009999998, 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049731319999999999, 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2217 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181751+0000", 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181752+0000", 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181952+0000", 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020039 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181963+0000", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0747000000000001e-05 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181968+0000", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3730000000000001e-06 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182010+0000", 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1737999999999999e-05 2024-09-18T01:07:40.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182094+0000", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3364999999999999e-05 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.182177+0000", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3053000000000006e-05 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183908+0000", 2024-09-18T01:07:40.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017312250000000001 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186667+0000", 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027587470000000002 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186700+0000", 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3642000000000001e-05 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186724+0000", 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3782999999999999e-05 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2273 3.1a 3:5df80eb9:::rbd_data.11769efc726f.0000000000000044:head [write 0~3312 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.0720153210000003, 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047637039999999997, 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2273 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191956+0000", 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.191957+0000", 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0299999999999997e-07 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192491+0000", 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053392200000000004 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192505+0000", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4428e-05 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192514+0000", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2349999999999999e-06 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192566+0000", 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2395000000000002e-05 2024-09-18T01:07:40.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192693+0000", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000126573 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.192845+0000", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015218799999999999 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.195787+0000", 2024-09-18T01:07:40.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029419540000000001 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196621+0000", 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083438100000000003 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196679+0000", 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7803000000000002e-05 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.196720+0000", 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0921999999999999e-05 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3082 3.1a 3:5e4f253b:::rbd_header.11769efc726f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.2569610139999998, 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044404190000000001, 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.320 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3082 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007011+0000", 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007012+0000", 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.604e-06 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007019+0000", 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3150000000000003e-06 2024-09-18T01:07:40.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007031+0000", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1676e-05 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007038+0000", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2790000000000001e-06 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007084+0000", 2024-09-18T01:07:40.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.007089+0000", 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3800000000000004e-06 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.009937+0000", 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002848363 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.010002+0000", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4672999999999996e-05 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.010363+0000", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036154399999999999 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011226+0000", 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000862618 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011390+0000", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016363200000000001 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011423+0000", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3368999999999998e-05 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.011451+0000", 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7948000000000001e-05 2024-09-18T01:07:40.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3111 3.3 3:c5270ba5:::rbd_data.11769efc726f.0000000000000045:head [write 327680~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0972220469999998, 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064345019999999999, 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3111 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166768+0000", 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8447000000000001e-05 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166774+0000", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5620000000000003e-06 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166777+0000", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1599999999999998e-06 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166827+0000", 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0037999999999998e-05 2024-09-18T01:07:40.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166926+0000", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8954000000000002e-05 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.167073+0000", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147317 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.169081+0000", 2024-09-18T01:07:40.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002007708 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173084+0000", 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040024220000000003 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173139+0000", 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5689999999999997e-05 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173184+0000", 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4821000000000003e-05 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3118 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'16, mlcod=29'14)", 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0846973310000001, 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048919489999999996, 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179274+0000", 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179275+0000", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4000000000000001e-07 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179282+0000", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1570000000000003e-06 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179291+0000", 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:07:40.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179302+0000", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0988000000000001e-05 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179338+0000", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6052e-05 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.179354+0000", 2024-09-18T01:07:40.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5452e-05 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184107+0000", 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047530009999999998 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184166+0000", 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9366e-05 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3119 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'17, mlcod=29'14)", 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0838766460000002, 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044026270000000001, 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:40.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180095+0000", 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5000000000000003e-07 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180100+0000", 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9080000000000003e-06 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180109+0000", 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1610000000000003e-06 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180113+0000", 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7350000000000004e-06 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180153+0000", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9233999999999997e-05 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.180165+0000", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2682e-05 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184457+0000", 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042922389999999998 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184498+0000", 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0117999999999998e-05 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3164 3.1a 3:59c1a3f5:::rbd_data.11769efc726f.0000000000000097:head [truncate 747215] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0434341700000003, 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045821480000000003, 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.342 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3164 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220537+0000", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220539+0000", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.443e-06 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220546+0000", 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1509999999999998e-06 2024-09-18T01:07:40.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220561+0000", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4555999999999999e-05 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220568+0000", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6040000000000004e-06 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220626+0000", 2024-09-18T01:07:40.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7821000000000001e-05 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220711+0000", 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5239000000000002e-05 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.220831+0000", 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011966100000000001 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.221532+0000", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000701288 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225056+0000", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035236970000000001 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225091+0000", 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5540999999999997e-05 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.225120+0000", 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8147e-05 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3232 3.9 3:91310cee:::rbd_data.11769efc726f.00000000000000bf:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9643940149999999, 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064668900000000003, 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3232 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6899999999999998e-07 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299582+0000", 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299590+0000", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000001e-06 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299595+0000", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300116+0000", 2024-09-18T01:07:40.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052083200000000004 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300194+0000", 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7818000000000006e-05 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300294+0000", 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100138 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.302169+0000", 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018753960000000001 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.305978+0000", 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038083610000000001 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306017+0000", 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9487000000000003e-05 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306044+0000", 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7367000000000001e-05 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3237 3.1b e29/23 3:de903dd5:::rbd_data.11769efc726f.00000000000000d6:head v 29'49, mlcod=29'47)", 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9636950080000002, 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057268120000000004, 2024-09-18T01:07:40.354 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300284+0000", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1130000000000003e-06 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300294+0000", 2024-09-18T01:07:40.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300305+0000", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.047e-05 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300353+0000", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8689000000000001e-05 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300375+0000", 2024-09-18T01:07:40.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1175999999999999e-05 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.305975+0000", 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005599924 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306003+0000", 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3261 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'56, mlcod=29'52)", 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.941269701, 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067026209999999998, 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322703+0000", 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000003e-07 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322708+0000", 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7169999999999996e-06 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322719+0000", 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1184e-05 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322726+0000", 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1310000000000003e-06 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322768+0000", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2706000000000001e-05 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322787+0000", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8428999999999999e-05 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329369+0000", 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065823870000000003 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329404+0000", 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5369e-05 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3263 3.1d e29/23 3:bf6d2803:::rbd_data.11769efc726f.00000000000000b1:head v 29'16, mlcod=29'14)", 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9403743220000003, 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058123380000000002, 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323598+0000", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323613+0000", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5325e-05 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323622+0000", 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999993e-06 2024-09-18T01:07:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323627+0000", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4020000000000004e-06 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323663+0000", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6356999999999998e-05 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323675+0000", 2024-09-18T01:07:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2254e-05 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329386+0000", 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005710283 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329410+0000", 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4051000000000001e-05 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3264 3.1b e29/23 3:dfb219e3:::rbd_data.11769efc726f.00000000000000b2:head v 29'57, mlcod=29'54)", 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9402790850000002, 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057289050000000003, 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323693+0000", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323704+0000", 2024-09-18T01:07:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0773e-05 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323710+0000", 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8300000000000001e-06 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323713+0000", 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3969999999999998e-06 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323747+0000", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.434e-05 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323763+0000", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.59e-05 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329407+0000", 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005643599 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329421+0000", 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4574e-05 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3278 3.14 3:2e019870:::rbd_data.11769efc726f.0000000000000042:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.347104+0000", 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.916867898, 2024-09-18T01:07:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0046726950000000001, 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3278 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347104+0000", 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347104+0000", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347104+0000", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347109+0000", 2024-09-18T01:07:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1019999999999996e-06 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347120+0000", 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0359e-05 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347125+0000", 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2979999999999999e-06 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347174+0000", 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9218000000000002e-05 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347258+0000", 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3514e-05 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.347581+0000", 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000323238 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.351505+0000", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039242069999999999 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.351690+0000", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018479999999999999 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.351736+0000", 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5776999999999999e-05 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.351776+0000", 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0451000000000003e-05 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.378 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.379 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:40.380 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:40.378+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:40.480 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:40.479+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:40.482 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.483 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:40.483 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:40.483 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.491 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:40.534 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:07:40.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:07:40.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:40.581 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:40.580+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:40.626 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.627 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:40.627 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:40.627 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.641 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:40.683 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:40.681+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:40.780 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:40.783 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:40.782+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:40.790 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7655076269999999, 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.791 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:40.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:07:40.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:07:40.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7654654760000001, 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.796 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:40.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:07:40.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7654320480000001, 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:40.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7653945479999997, 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:40.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7653729709999997, 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:07:40.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7653538309999997, 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.815 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.819 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.76533344, 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:40.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7653040400000002, 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:07:40.824 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:07:40.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:07:40.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7652752500000002, 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:07:40.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.833 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7652545550000003, 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:07:40.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7652350710000002, 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:07:40.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7652011600000002, 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.843 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:07:40.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7651651609999996, 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:07:40.847 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:07:40.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6807171890000001, 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.852 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:07:40.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6807069800000001, 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.857 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:07:40.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.18030575400000001, 2024-09-18T01:07:40.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:40.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:07:40.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:07:40.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.16481958499999999, 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:07:40.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:07:40.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:40.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.156928865, 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:07:40.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13085986199999999, 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.877 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:07:40.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:40.883 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12569106999999999, 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:07:40.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:40.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:40.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:40.888 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:40.889 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:40.887+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:40.890 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:40.902 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:40.921 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:40.925 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:40.926 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:40.942 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-18T01:07:40.725250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110971291,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":5138,"num_read_kb":48435,"num_write":1184,"num_write_kb":47768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314584,"kb_used_data":77432,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754660136,"statfs":{"total":773094113280,"available":772771979264,"internally_reserved":0,"allocated":79290368,"data_stored":77385030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":86241465,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":4367,"num_read_kb":47741,"num_write":827,"num_write_kb":38765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.708792"},"pg_stats":[{"pgid":"3.1f","version":"29'10","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024432+0000","last_change":"2024-09-18T01:07:36.014676+0000","last_active":"2024-09-18T01:07:37.024432+0000","last_peered":"2024-09-18T01:07:37.024432+0000","last_clean":"2024-09-18T01:07:37.024432+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:07:37.024432+0000","last_undegraded":"2024-09-18T01:07:37.024432+0000","last_fullsized":"2024-09-18T01:07:37.024432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:56.237860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000663257,"stat_sum":{"num_bytes":1769488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":8,"num_write":12,"num_write_kb":1122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1e","version":"31'17","reported_seq":182,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.542039+0000","last_change":"2024-09-18T01:07:39.033643+0000","last_active":"2024-09-18T01:07:39.542039+0000","last_peered":"2024-09-18T01:07:39.542039+0000","last_clean":"2024-09-18T01:07:39.542039+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:07:39.542039+0000","last_undegraded":"2024-09-18T01:07:39.542039+0000","last_fullsized":"2024-09-18T01:07:39.542039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:10.965284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079123799999999999,"stat_sum":{"num_bytes":6535985,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":2307,"num_write":25,"num_write_kb":3793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.1d","version":"29'30","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028542+0000","last_change":"2024-09-18T01:07:36.174208+0000","last_active":"2024-09-18T01:07:37.028542+0000","last_peered":"2024-09-18T01:07:37.028542+0000","last_clean":"2024-09-18T01:07:37.028542+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:07:37.028542+0000","last_undegraded":"2024-09-18T01:07:37.028542+0000","last_fullsized":"2024-09-18T01:07:37.028542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:01.037251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065585199999999995,"stat_sum":{"num_bytes":5538980,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":219,"num_read_kb":3391,"num_write":32,"num_write_kb":2149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1c","version":"30'8","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028499+0000","last_change":"2024-09-18T01:07:36.174138+0000","last_active":"2024-09-18T01:07:37.028499+0000","last_peered":"2024-09-18T01:07:37.028499+0000","last_clean":"2024-09-18T01:07:37.028499+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:07:37.028499+0000","last_undegraded":"2024-09-18T01:07:37.028499+0000","last_fullsized":"2024-09-18T01:07:37.028499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:14.005330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054387800000000003,"stat_sum":{"num_bytes":1585331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1b","version":"30'124","reported_seq":457,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.031114+0000","last_change":"2024-09-18T01:07:36.015218+0000","last_active":"2024-09-18T01:07:37.031114+0000","last_peered":"2024-09-18T01:07:37.031114+0000","last_clean":"2024-09-18T01:07:37.031114+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:07:37.031114+0000","last_undegraded":"2024-09-18T01:07:37.031114+0000","last_fullsized":"2024-09-18T01:07:37.031114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:03.022782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001044618,"stat_sum":{"num_bytes":5380009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":817,"num_read_kb":4849,"num_write":200,"num_write_kb":1581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1a","version":"28'21","reported_seq":127,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.339874+0000","last_change":"2024-09-18T01:07:30.977745+0000","last_active":"2024-09-18T01:07:35.339874+0000","last_peered":"2024-09-18T01:07:35.339874+0000","last_clean":"2024-09-18T01:07:35.339874+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:07:35.339874+0000","last_undegraded":"2024-09-18T01:07:35.339874+0000","last_fullsized":"2024-09-18T01:07:35.339874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:08:41.946938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006018,"stat_sum":{"num_bytes":3067139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":156,"num_read_kb":192,"num_write":24,"num_write_kb":1553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"29'16","reported_seq":320,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017952+0000","last_change":"2024-09-18T01:07:36.006610+0000","last_active":"2024-09-18T01:07:37.017952+0000","last_peered":"2024-09-18T01:07:37.017952+0000","last_clean":"2024-09-18T01:07:37.017952+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:07:37.017952+0000","last_undegraded":"2024-09-18T01:07:37.017952+0000","last_fullsized":"2024-09-18T01:07:37.017952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:29.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050978600000000003,"stat_sum":{"num_bytes":5517562,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":269,"num_read_kb":4157,"num_write":18,"num_write_kb":1543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.18","version":"32'5","reported_seq":134,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541502+0000","last_change":"2024-09-18T01:07:39.033596+0000","last_active":"2024-09-18T01:07:39.541502+0000","last_peered":"2024-09-18T01:07:39.541502+0000","last_clean":"2024-09-18T01:07:39.541502+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:07:39.541502+0000","last_undegraded":"2024-09-18T01:07:39.541502+0000","last_fullsized":"2024-09-18T01:07:39.541502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:50.228073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079001899999999999,"stat_sum":{"num_bytes":2748373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":1952,"num_write":5,"num_write_kb":951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.17","version":"29'17","reported_seq":111,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024554+0000","last_change":"2024-09-18T01:07:36.014679+0000","last_active":"2024-09-18T01:07:37.024554+0000","last_peered":"2024-09-18T01:07:37.024554+0000","last_clean":"2024-09-18T01:07:37.024554+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:07:37.024554+0000","last_undegraded":"2024-09-18T01:07:37.024554+0000","last_fullsized":"2024-09-18T01:07:37.024554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:47.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058556099999999996,"stat_sum":{"num_bytes":3731297,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":786,"num_write":18,"num_write_kb":1656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.16","version":"29'8","reported_seq":208,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028492+0000","last_change":"2024-09-18T01:07:36.174051+0000","last_active":"2024-09-18T01:07:37.028492+0000","last_peered":"2024-09-18T01:07:37.028492+0000","last_clean":"2024-09-18T01:07:37.028492+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:07:37.028492+0000","last_undegraded":"2024-09-18T01:07:37.028492+0000","last_fullsized":"2024-09-18T01:07:37.028492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:31:11.463031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044398199999999999,"stat_sum":{"num_bytes":4900874,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":167,"num_read_kb":2532,"num_write":9,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.15","version":"30'21","reported_seq":328,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.068389+0000","last_change":"2024-09-18T01:07:36.178507+0000","last_active":"2024-09-18T01:07:37.068389+0000","last_peered":"2024-09-18T01:07:37.068389+0000","last_clean":"2024-09-18T01:07:37.068389+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:07:37.068389+0000","last_undegraded":"2024-09-18T01:07:37.068389+0000","last_fullsized":"2024-09-18T01:07:37.068389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:02.721617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067200199999999999,"stat_sum":{"num_bytes":6229819,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":294,"num_read_kb":3897,"num_write":21,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.14","version":"28'77","reported_seq":104,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.183972+0000","last_change":"2024-09-18T01:07:30.978015+0000","last_active":"2024-09-18T01:07:35.183972+0000","last_peered":"2024-09-18T01:07:35.183972+0000","last_clean":"2024-09-18T01:07:35.183972+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:07:35.183972+0000","last_undegraded":"2024-09-18T01:07:35.183972+0000","last_fullsized":"2024-09-18T01:07:35.183972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:49.193210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235795,"stat_sum":{"num_bytes":2687025,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":357,"num_read_kb":285,"num_write":126,"num_write_kb":1734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.053571+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:37.053571+0000","last_peered":"2024-09-18T01:07:37.053571+0000","last_clean":"2024-09-18T01:07:37.053571+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:37.053571+0000","last_undegraded":"2024-09-18T01:07:37.053571+0000","last_fullsized":"2024-09-18T01:07:37.053571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'31","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024449+0000","last_change":"2024-09-18T01:07:36.014979+0000","last_active":"2024-09-18T01:07:37.024449+0000","last_peered":"2024-09-18T01:07:37.024449+0000","last_clean":"2024-09-18T01:07:37.024449+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:07:37.024449+0000","last_undegraded":"2024-09-18T01:07:37.024449+0000","last_fullsized":"2024-09-18T01:07:37.024449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:38.195081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072934200000000001,"stat_sum":{"num_bytes":2320351,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":525,"num_read_kb":1490,"num_write":29,"num_write_kb":182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017887+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:37.017887+0000","last_peered":"2024-09-18T01:07:37.017887+0000","last_clean":"2024-09-18T01:07:37.017887+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:37.017887+0000","last_undegraded":"2024-09-18T01:07:37.017887+0000","last_fullsized":"2024-09-18T01:07:37.017887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'130","reported_seq":209,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541775+0000","last_change":"2024-09-18T01:07:39.033065+0000","last_active":"2024-09-18T01:07:39.541775+0000","last_peered":"2024-09-18T01:07:39.541775+0000","last_clean":"2024-09-18T01:07:39.541775+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:07:39.541775+0000","last_undegraded":"2024-09-18T01:07:39.541775+0000","last_fullsized":"2024-09-18T01:07:39.541775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:43.814814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045197,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":635,"num_read_kb":1260,"num_write":255,"num_write_kb":3759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018118+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:37.018118+0000","last_peered":"2024-09-18T01:07:37.018118+0000","last_clean":"2024-09-18T01:07:37.018118+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:37.018118+0000","last_undegraded":"2024-09-18T01:07:37.018118+0000","last_fullsized":"2024-09-18T01:07:37.018118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"29'34","reported_seq":520,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017843+0000","last_change":"2024-09-18T01:07:36.006726+0000","last_active":"2024-09-18T01:07:37.017843+0000","last_peered":"2024-09-18T01:07:37.017843+0000","last_clean":"2024-09-18T01:07:37.017843+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:07:37.017843+0000","last_undegraded":"2024-09-18T01:07:37.017843+0000","last_fullsized":"2024-09-18T01:07:37.017843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:15.344375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035619100000000001,"stat_sum":{"num_bytes":11658758,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":451,"num_read_kb":6652,"num_write":36,"num_write_kb":2407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017823+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:37.017823+0000","last_peered":"2024-09-18T01:07:37.017823+0000","last_clean":"2024-09-18T01:07:37.017823+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:37.017823+0000","last_undegraded":"2024-09-18T01:07:37.017823+0000","last_fullsized":"2024-09-18T01:07:37.017823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"29'4","reported_seq":166,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028585+0000","last_change":"2024-09-18T01:07:36.174754+0000","last_active":"2024-09-18T01:07:37.028585+0000","last_peered":"2024-09-18T01:07:37.028585+0000","last_clean":"2024-09-18T01:07:37.028585+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:07:37.028585+0000","last_undegraded":"2024-09-18T01:07:37.028585+0000","last_fullsized":"2024-09-18T01:07:37.028585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:59:48.227811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162405,"stat_sum":{"num_bytes":2356712,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":1934,"num_write":4,"num_write_kb":411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.3","version":"28'5","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.201211+0000","last_change":"2024-09-18T01:07:30.977158+0000","last_active":"2024-09-18T01:07:35.201211+0000","last_peered":"2024-09-18T01:07:35.201211+0000","last_clean":"2024-09-18T01:07:35.201211+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:07:35.201211+0000","last_undegraded":"2024-09-18T01:07:35.201211+0000","last_fullsized":"2024-09-18T01:07:35.201211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:22.451886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042770500000000001,"stat_sum":{"num_bytes":3932160,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028339+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:37.028339+0000","last_peered":"2024-09-18T01:07:37.028339+0000","last_clean":"2024-09-18T01:07:37.028339+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:37.028339+0000","last_undegraded":"2024-09-18T01:07:37.028339+0000","last_fullsized":"2024-09-18T01:07:37.028339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"30'14","reported_seq":56,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.057365+0000","last_change":"2024-09-18T01:07:36.006240+0000","last_active":"2024-09-18T01:07:37.057365+0000","last_peered":"2024-09-18T01:07:37.057365+0000","last_clean":"2024-09-18T01:07:37.057365+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:07:37.057365+0000","last_undegraded":"2024-09-18T01:07:37.057365+0000","last_fullsized":"2024-09-18T01:07:37.057365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:12.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040466299999999998,"stat_sum":{"num_bytes":7126787,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":12,"num_read_kb":22,"num_write":15,"num_write_kb":3453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017030+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:37.017030+0000","last_peered":"2024-09-18T01:07:37.017030+0000","last_clean":"2024-09-18T01:07:37.017030+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:37.017030+0000","last_undegraded":"2024-09-18T01:07:37.017030+0000","last_fullsized":"2024-09-18T01:07:37.017030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"29'9","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024366+0000","last_change":"2024-09-18T01:07:36.014344+0000","last_active":"2024-09-18T01:07:37.024366+0000","last_peered":"2024-09-18T01:07:37.024366+0000","last_clean":"2024-09-18T01:07:37.024366+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:07:37.024366+0000","last_undegraded":"2024-09-18T01:07:37.024366+0000","last_fullsized":"2024-09-18T01:07:37.024366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:34.245986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039254699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017841+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:37.017841+0000","last_peered":"2024-09-18T01:07:37.017841+0000","last_clean":"2024-09-18T01:07:37.017841+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:37.017841+0000","last_undegraded":"2024-09-18T01:07:37.017841+0000","last_fullsized":"2024-09-18T01:07:37.017841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"33'18","reported_seq":73,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.548491+0000","last_change":"2024-09-18T01:07:39.033017+0000","last_active":"2024-09-18T01:07:39.548491+0000","last_peered":"2024-09-18T01:07:39.548491+0000","last_clean":"2024-09-18T01:07:39.548491+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:07:39.548491+0000","last_undegraded":"2024-09-18T01:07:39.548491+0000","last_fullsized":"2024-09-18T01:07:39.548491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:54.683584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040146600000000001,"stat_sum":{"num_bytes":138,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":6,"num_write":18,"num_write_kb":506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028359+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:37.028359+0000","last_peered":"2024-09-18T01:07:37.028359+0000","last_clean":"2024-09-18T01:07:37.028359+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:37.028359+0000","last_undegraded":"2024-09-18T01:07:37.028359+0000","last_fullsized":"2024-09-18T01:07:37.028359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017852+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:37.017852+0000","last_peered":"2024-09-18T01:07:37.017852+0000","last_clean":"2024-09-18T01:07:37.017852+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:37.017852+0000","last_undegraded":"2024-09-18T01:07:37.017852+0000","last_fullsized":"2024-09-18T01:07:37.017852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'18","reported_seq":214,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541776+0000","last_change":"2024-09-18T01:07:39.033353+0000","last_active":"2024-09-18T01:07:39.541776+0000","last_peered":"2024-09-18T01:07:39.541776+0000","last_clean":"2024-09-18T01:07:39.541776+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:07:39.541776+0000","last_undegraded":"2024-09-18T01:07:39.541776+0000","last_fullsized":"2024-09-18T01:07:39.541776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:08.105385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055374899999999997,"stat_sum":{"num_bytes":3686400,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":3632,"num_write":19,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.9","version":"28'6","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.143163+0000","last_change":"2024-09-18T01:07:30.977803+0000","last_active":"2024-09-18T01:07:35.143163+0000","last_peered":"2024-09-18T01:07:35.143163+0000","last_clean":"2024-09-18T01:07:35.143163+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:07:35.143163+0000","last_undegraded":"2024-09-18T01:07:35.143163+0000","last_fullsized":"2024-09-18T01:07:35.143163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:07:20.080474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060928,"stat_sum":{"num_bytes":3726593,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"29'14","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.053629+0000","last_change":"2024-09-18T01:07:36.006028+0000","last_active":"2024-09-18T01:07:37.053629+0000","last_peered":"2024-09-18T01:07:37.053629+0000","last_clean":"2024-09-18T01:07:37.053629+0000","last_became_active":"2024-09-18T01:07:28.974981+0000","last_became_peered":"2024-09-18T01:07:28.974981+0000","last_unstale":"2024-09-18T01:07:37.053629+0000","last_undegraded":"2024-09-18T01:07:37.053629+0000","last_fullsized":"2024-09-18T01:07:37.053629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:55:29.455496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040640199999999999,"stat_sum":{"num_bytes":2983076,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":16,"num_write_kb":1657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.b","version":"31'19","reported_seq":383,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541815+0000","last_change":"2024-09-18T01:07:39.033402+0000","last_active":"2024-09-18T01:07:39.541815+0000","last_peered":"2024-09-18T01:07:39.541815+0000","last_clean":"2024-09-18T01:07:39.541815+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:07:39.541815+0000","last_undegraded":"2024-09-18T01:07:39.541815+0000","last_fullsized":"2024-09-18T01:07:39.541815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:04.373023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072990499999999999,"stat_sum":{"num_bytes":9133313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":318,"num_read_kb":5645,"num_write":29,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.c","version":"29'8","reported_seq":233,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028452+0000","last_change":"2024-09-18T01:07:36.174538+0000","last_active":"2024-09-18T01:07:37.028452+0000","last_peered":"2024-09-18T01:07:37.028452+0000","last_clean":"2024-09-18T01:07:37.028452+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:07:37.028452+0000","last_undegraded":"2024-09-18T01:07:37.028452+0000","last_fullsized":"2024-09-18T01:07:37.028452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:49.390029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097729499999999999,"stat_sum":{"num_bytes":3615160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2939,"num_write":8,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.d","version":"29'11","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018085+0000","last_change":"2024-09-18T01:07:36.178675+0000","last_active":"2024-09-18T01:07:37.018085+0000","last_peered":"2024-09-18T01:07:37.018085+0000","last_clean":"2024-09-18T01:07:37.018085+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:07:37.018085+0000","last_undegraded":"2024-09-18T01:07:37.018085+0000","last_fullsized":"2024-09-18T01:07:37.018085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:55.826275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057819999999999996,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":5,"num_write":11,"num_write_kb":1019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.e","version":"29'6","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018222+0000","last_change":"2024-09-18T01:07:36.178175+0000","last_active":"2024-09-18T01:07:37.018222+0000","last_peered":"2024-09-18T01:07:37.018222+0000","last_clean":"2024-09-18T01:07:37.018222+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:07:37.018222+0000","last_undegraded":"2024-09-18T01:07:37.018222+0000","last_fullsized":"2024-09-18T01:07:37.018222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:15.200273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037671100000000001,"stat_sum":{"num_bytes":1884160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":16,"num_write":7,"num_write_kb":1130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.f","version":"29'3","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018293+0000","last_change":"2024-09-18T01:07:36.178713+0000","last_active":"2024-09-18T01:07:37.018293+0000","last_peered":"2024-09-18T01:07:37.018293+0000","last_clean":"2024-09-18T01:07:37.018293+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:07:37.018293+0000","last_undegraded":"2024-09-18T01:07:37.018293+0000","last_fullsized":"2024-09-18T01:07:37.018293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:25.935817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062224000000000003,"stat_sum":{"num_bytes":753664,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.10","version":"30'10","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.067551+0000","last_change":"2024-09-18T01:07:36.005973+0000","last_active":"2024-09-18T01:07:37.067551+0000","last_peered":"2024-09-18T01:07:37.067551+0000","last_clean":"2024-09-18T01:07:37.067551+0000","last_became_active":"2024-09-18T01:07:28.976873+0000","last_became_peered":"2024-09-18T01:07:28.976873+0000","last_unstale":"2024-09-18T01:07:37.067551+0000","last_undegraded":"2024-09-18T01:07:37.067551+0000","last_fullsized":"2024-09-18T01:07:37.067551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:14.481178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000360873,"stat_sum":{"num_bytes":702356,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":10,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.11","version":"29'6","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017944+0000","last_change":"2024-09-18T01:07:36.178074+0000","last_active":"2024-09-18T01:07:37.017944+0000","last_peered":"2024-09-18T01:07:37.017944+0000","last_clean":"2024-09-18T01:07:37.017944+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:07:37.017944+0000","last_undegraded":"2024-09-18T01:07:37.017944+0000","last_fullsized":"2024-09-18T01:07:37.017944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:03.904405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034916500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.12","version":"29'17","reported_seq":84,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024348+0000","last_change":"2024-09-18T01:07:36.014355+0000","last_active":"2024-09-18T01:07:37.024348+0000","last_peered":"2024-09-18T01:07:37.024348+0000","last_clean":"2024-09-18T01:07:37.024348+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:07:37.024348+0000","last_undegraded":"2024-09-18T01:07:37.024348+0000","last_fullsized":"2024-09-18T01:07:37.024348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:04:54.832703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000392972,"stat_sum":{"num_bytes":1812274,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":367,"num_write":17,"num_write_kb":631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.13","version":"29'15","reported_seq":50,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017985+0000","last_change":"2024-09-18T01:07:36.178293+0000","last_active":"2024-09-18T01:07:37.017985+0000","last_peered":"2024-09-18T01:07:37.017985+0000","last_clean":"2024-09-18T01:07:37.017985+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:07:37.017985+0000","last_undegraded":"2024-09-18T01:07:37.017985+0000","last_fullsized":"2024-09-18T01:07:37.017985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:42.884146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049598600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":110380904,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":5021,"num_read_kb":48334,"num_write":1023,"num_write_kb":45350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":75542528,"data_stored":75197448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":732,"ondisk_log_size":732,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37492,"kb_used_data":6276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334348,"statfs":{"total":96636764160,"available":96598372352,"internally_reserved":0,"allocated":6426624,"data_stored":6209367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29520,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342320,"statfs":{"total":96636764160,"available":96606535680,"internally_reserved":0,"allocated":2531328,"data_stored":2323673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":5528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":5660672,"data_stored":5419826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34180,"kb_used_data":7144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337660,"statfs":{"total":96636764160,"available":96601763840,"internally_reserved":0,"allocated":7315456,"data_stored":7089168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40660,"kb_used_data":9460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331180,"statfs":{"total":96636764160,"available":96595128320,"internally_reserved":0,"allocated":9687040,"data_stored":9413434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39540,"kb_used_data":8308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332300,"statfs":{"total":96636764160,"available":96596275200,"internally_reserved":0,"allocated":8507392,"data_stored":8267405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45172,"kb_used_data":13980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326668,"statfs":{"total":96636764160,"available":96590508032,"internally_reserved":0,"allocated":14315520,"data_stored":14088230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55456,"kb_used_data":24264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316384,"statfs":{"total":96636764160,"available":96579977216,"internally_reserved":0,"allocated":24846336,"data_stored":24573927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7593984,"data_stored":7551193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9363456,"data_stored":9287571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7012352,"data_stored":6969503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24489984,"data_stored":24429466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14020608,"data_stored":13974768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5337088,"data_stored":5293963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2211840,"data_stored":2197829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5513216,"data_stored":5493155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:40.944 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:40.945 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:40.946 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:40.961 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:40.961 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:40.978 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:40.978 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:40.990 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:40.989+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:40.997 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:41.014 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:41.014 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:41.032 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:41.032 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:41.038 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.039 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:41.091 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:41.090+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:41.192 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:41.191+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:41.193 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.193 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.202 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:41.224 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:41.235 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-18T01:07:40.725250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110971291,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":5138,"num_read_kb":48435,"num_write":1184,"num_write_kb":47768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314584,"kb_used_data":77432,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754660136,"statfs":{"total":773094113280,"available":772771979264,"internally_reserved":0,"allocated":79290368,"data_stored":77385030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":86241465,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":4367,"num_read_kb":47741,"num_write":827,"num_write_kb":38765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.708792"},"pg_stats":[{"pgid":"3.1f","version":"29'10","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024432+0000","last_change":"2024-09-18T01:07:36.014676+0000","last_active":"2024-09-18T01:07:37.024432+0000","last_peered":"2024-09-18T01:07:37.024432+0000","last_clean":"2024-09-18T01:07:37.024432+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:07:37.024432+0000","last_undegraded":"2024-09-18T01:07:37.024432+0000","last_fullsized":"2024-09-18T01:07:37.024432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:56.237860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000663257,"stat_sum":{"num_bytes":1769488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":8,"num_write":12,"num_write_kb":1122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1e","version":"31'17","reported_seq":182,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.542039+0000","last_change":"2024-09-18T01:07:39.033643+0000","last_active":"2024-09-18T01:07:39.542039+0000","last_peered":"2024-09-18T01:07:39.542039+0000","last_clean":"2024-09-18T01:07:39.542039+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:07:39.542039+0000","last_undegraded":"2024-09-18T01:07:39.542039+0000","last_fullsized":"2024-09-18T01:07:39.542039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:10.965284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079123799999999999,"stat_sum":{"num_bytes":6535985,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":2307,"num_write":25,"num_write_kb":3793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.1d","version":"29'30","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028542+0000","last_change":"2024-09-18T01:07:36.174208+0000","last_active":"2024-09-18T01:07:37.028542+0000","last_peered":"2024-09-18T01:07:37.028542+0000","last_clean":"2024-09-18T01:07:37.028542+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:07:37.028542+0000","last_undegraded":"2024-09-18T01:07:37.028542+0000","last_fullsized":"2024-09-18T01:07:37.028542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:01.037251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065585199999999995,"stat_sum":{"num_bytes":5538980,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":219,"num_read_kb":3391,"num_write":32,"num_write_kb":2149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1c","version":"30'8","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028499+0000","last_change":"2024-09-18T01:07:36.174138+0000","last_active":"2024-09-18T01:07:37.028499+0000","last_peered":"2024-09-18T01:07:37.028499+0000","last_clean":"2024-09-18T01:07:37.028499+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:07:37.028499+0000","last_undegraded":"2024-09-18T01:07:37.028499+0000","last_fullsized":"2024-09-18T01:07:37.028499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:14.005330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054387800000000003,"stat_sum":{"num_bytes":1585331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1b","version":"30'124","reported_seq":457,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.031114+0000","last_change":"2024-09-18T01:07:36.015218+0000","last_active":"2024-09-18T01:07:37.031114+0000","last_peered":"2024-09-18T01:07:37.031114+0000","last_clean":"2024-09-18T01:07:37.031114+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:07:37.031114+0000","last_undegraded":"2024-09-18T01:07:37.031114+0000","last_fullsized":"2024-09-18T01:07:37.031114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:03.022782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001044618,"stat_sum":{"num_bytes":5380009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":817,"num_read_kb":4849,"num_write":200,"num_write_kb":1581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.1a","version":"28'21","reported_seq":127,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.339874+0000","last_change":"2024-09-18T01:07:30.977745+0000","last_active":"2024-09-18T01:07:35.339874+0000","last_peered":"2024-09-18T01:07:35.339874+0000","last_clean":"2024-09-18T01:07:35.339874+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:07:35.339874+0000","last_undegraded":"2024-09-18T01:07:35.339874+0000","last_fullsized":"2024-09-18T01:07:35.339874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:08:41.946938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006018,"stat_sum":{"num_bytes":3067139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":156,"num_read_kb":192,"num_write":24,"num_write_kb":1553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"29'16","reported_seq":320,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017952+0000","last_change":"2024-09-18T01:07:36.006610+0000","last_active":"2024-09-18T01:07:37.017952+0000","last_peered":"2024-09-18T01:07:37.017952+0000","last_clean":"2024-09-18T01:07:37.017952+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:07:37.017952+0000","last_undegraded":"2024-09-18T01:07:37.017952+0000","last_fullsized":"2024-09-18T01:07:37.017952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:29.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050978600000000003,"stat_sum":{"num_bytes":5517562,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":269,"num_read_kb":4157,"num_write":18,"num_write_kb":1543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.18","version":"32'5","reported_seq":134,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541502+0000","last_change":"2024-09-18T01:07:39.033596+0000","last_active":"2024-09-18T01:07:39.541502+0000","last_peered":"2024-09-18T01:07:39.541502+0000","last_clean":"2024-09-18T01:07:39.541502+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:07:39.541502+0000","last_undegraded":"2024-09-18T01:07:39.541502+0000","last_fullsized":"2024-09-18T01:07:39.541502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:50.228073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079001899999999999,"stat_sum":{"num_bytes":2748373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":1952,"num_write":5,"num_write_kb":951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.17","version":"29'17","reported_seq":111,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024554+0000","last_change":"2024-09-18T01:07:36.014679+0000","last_active":"2024-09-18T01:07:37.024554+0000","last_peered":"2024-09-18T01:07:37.024554+0000","last_clean":"2024-09-18T01:07:37.024554+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:07:37.024554+0000","last_undegraded":"2024-09-18T01:07:37.024554+0000","last_fullsized":"2024-09-18T01:07:37.024554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:47.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058556099999999996,"stat_sum":{"num_bytes":3731297,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":786,"num_write":18,"num_write_kb":1656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.16","version":"29'8","reported_seq":208,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028492+0000","last_change":"2024-09-18T01:07:36.174051+0000","last_active":"2024-09-18T01:07:37.028492+0000","last_peered":"2024-09-18T01:07:37.028492+0000","last_clean":"2024-09-18T01:07:37.028492+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:07:37.028492+0000","last_undegraded":"2024-09-18T01:07:37.028492+0000","last_fullsized":"2024-09-18T01:07:37.028492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:31:11.463031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044398199999999999,"stat_sum":{"num_bytes":4900874,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":167,"num_read_kb":2532,"num_write":9,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.15","version":"30'21","reported_seq":328,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.068389+0000","last_change":"2024-09-18T01:07:36.178507+0000","last_active":"2024-09-18T01:07:37.068389+0000","last_peered":"2024-09-18T01:07:37.068389+0000","last_clean":"2024-09-18T01:07:37.068389+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:07:37.068389+0000","last_undegraded":"2024-09-18T01:07:37.068389+0000","last_fullsized":"2024-09-18T01:07:37.068389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:02.721617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067200199999999999,"stat_sum":{"num_bytes":6229819,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":294,"num_read_kb":3897,"num_write":21,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.14","version":"28'77","reported_seq":104,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.183972+0000","last_change":"2024-09-18T01:07:30.978015+0000","last_active":"2024-09-18T01:07:35.183972+0000","last_peered":"2024-09-18T01:07:35.183972+0000","last_clean":"2024-09-18T01:07:35.183972+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:07:35.183972+0000","last_undegraded":"2024-09-18T01:07:35.183972+0000","last_fullsized":"2024-09-18T01:07:35.183972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:49.193210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235795,"stat_sum":{"num_bytes":2687025,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":357,"num_read_kb":285,"num_write":126,"num_write_kb":1734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.053571+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:37.053571+0000","last_peered":"2024-09-18T01:07:37.053571+0000","last_clean":"2024-09-18T01:07:37.053571+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:37.053571+0000","last_undegraded":"2024-09-18T01:07:37.053571+0000","last_fullsized":"2024-09-18T01:07:37.053571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'31","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024449+0000","last_change":"2024-09-18T01:07:36.014979+0000","last_active":"2024-09-18T01:07:37.024449+0000","last_peered":"2024-09-18T01:07:37.024449+0000","last_clean":"2024-09-18T01:07:37.024449+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:07:37.024449+0000","last_undegraded":"2024-09-18T01:07:37.024449+0000","last_fullsized":"2024-09-18T01:07:37.024449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:38.195081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072934200000000001,"stat_sum":{"num_bytes":2320351,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":525,"num_read_kb":1490,"num_write":29,"num_write_kb":182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017887+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:37.017887+0000","last_peered":"2024-09-18T01:07:37.017887+0000","last_clean":"2024-09-18T01:07:37.017887+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:37.017887+0000","last_undegraded":"2024-09-18T01:07:37.017887+0000","last_fullsized":"2024-09-18T01:07:37.017887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'130","reported_seq":209,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541775+0000","last_change":"2024-09-18T01:07:39.033065+0000","last_active":"2024-09-18T01:07:39.541775+0000","last_peered":"2024-09-18T01:07:39.541775+0000","last_clean":"2024-09-18T01:07:39.541775+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:07:39.541775+0000","last_undegraded":"2024-09-18T01:07:39.541775+0000","last_fullsized":"2024-09-18T01:07:39.541775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:43.814814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045197,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":635,"num_read_kb":1260,"num_write":255,"num_write_kb":3759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018118+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:37.018118+0000","last_peered":"2024-09-18T01:07:37.018118+0000","last_clean":"2024-09-18T01:07:37.018118+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:37.018118+0000","last_undegraded":"2024-09-18T01:07:37.018118+0000","last_fullsized":"2024-09-18T01:07:37.018118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"29'34","reported_seq":520,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017843+0000","last_change":"2024-09-18T01:07:36.006726+0000","last_active":"2024-09-18T01:07:37.017843+0000","last_peered":"2024-09-18T01:07:37.017843+0000","last_clean":"2024-09-18T01:07:37.017843+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:07:37.017843+0000","last_undegraded":"2024-09-18T01:07:37.017843+0000","last_fullsized":"2024-09-18T01:07:37.017843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:15.344375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035619100000000001,"stat_sum":{"num_bytes":11658758,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":451,"num_read_kb":6652,"num_write":36,"num_write_kb":2407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017823+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:37.017823+0000","last_peered":"2024-09-18T01:07:37.017823+0000","last_clean":"2024-09-18T01:07:37.017823+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:37.017823+0000","last_undegraded":"2024-09-18T01:07:37.017823+0000","last_fullsized":"2024-09-18T01:07:37.017823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"29'4","reported_seq":166,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028585+0000","last_change":"2024-09-18T01:07:36.174754+0000","last_active":"2024-09-18T01:07:37.028585+0000","last_peered":"2024-09-18T01:07:37.028585+0000","last_clean":"2024-09-18T01:07:37.028585+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:07:37.028585+0000","last_undegraded":"2024-09-18T01:07:37.028585+0000","last_fullsized":"2024-09-18T01:07:37.028585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:59:48.227811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162405,"stat_sum":{"num_bytes":2356712,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":1934,"num_write":4,"num_write_kb":411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.3","version":"28'5","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.201211+0000","last_change":"2024-09-18T01:07:30.977158+0000","last_active":"2024-09-18T01:07:35.201211+0000","last_peered":"2024-09-18T01:07:35.201211+0000","last_clean":"2024-09-18T01:07:35.201211+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:07:35.201211+0000","last_undegraded":"2024-09-18T01:07:35.201211+0000","last_fullsized":"2024-09-18T01:07:35.201211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:22.451886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042770500000000001,"stat_sum":{"num_bytes":3932160,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028339+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:37.028339+0000","last_peered":"2024-09-18T01:07:37.028339+0000","last_clean":"2024-09-18T01:07:37.028339+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:37.028339+0000","last_undegraded":"2024-09-18T01:07:37.028339+0000","last_fullsized":"2024-09-18T01:07:37.028339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"30'14","reported_seq":56,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.057365+0000","last_change":"2024-09-18T01:07:36.006240+0000","last_active":"2024-09-18T01:07:37.057365+0000","last_peered":"2024-09-18T01:07:37.057365+0000","last_clean":"2024-09-18T01:07:37.057365+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:07:37.057365+0000","last_undegraded":"2024-09-18T01:07:37.057365+0000","last_fullsized":"2024-09-18T01:07:37.057365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:12.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040466299999999998,"stat_sum":{"num_bytes":7126787,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":12,"num_read_kb":22,"num_write":15,"num_write_kb":3453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017030+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:37.017030+0000","last_peered":"2024-09-18T01:07:37.017030+0000","last_clean":"2024-09-18T01:07:37.017030+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:37.017030+0000","last_undegraded":"2024-09-18T01:07:37.017030+0000","last_fullsized":"2024-09-18T01:07:37.017030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"29'9","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024366+0000","last_change":"2024-09-18T01:07:36.014344+0000","last_active":"2024-09-18T01:07:37.024366+0000","last_peered":"2024-09-18T01:07:37.024366+0000","last_clean":"2024-09-18T01:07:37.024366+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:07:37.024366+0000","last_undegraded":"2024-09-18T01:07:37.024366+0000","last_fullsized":"2024-09-18T01:07:37.024366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:34.245986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039254699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017841+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:37.017841+0000","last_peered":"2024-09-18T01:07:37.017841+0000","last_clean":"2024-09-18T01:07:37.017841+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:37.017841+0000","last_undegraded":"2024-09-18T01:07:37.017841+0000","last_fullsized":"2024-09-18T01:07:37.017841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"33'18","reported_seq":73,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.548491+0000","last_change":"2024-09-18T01:07:39.033017+0000","last_active":"2024-09-18T01:07:39.548491+0000","last_peered":"2024-09-18T01:07:39.548491+0000","last_clean":"2024-09-18T01:07:39.548491+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:07:39.548491+0000","last_undegraded":"2024-09-18T01:07:39.548491+0000","last_fullsized":"2024-09-18T01:07:39.548491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:54.683584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040146600000000001,"stat_sum":{"num_bytes":138,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":6,"num_write":18,"num_write_kb":506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028359+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:37.028359+0000","last_peered":"2024-09-18T01:07:37.028359+0000","last_clean":"2024-09-18T01:07:37.028359+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:37.028359+0000","last_undegraded":"2024-09-18T01:07:37.028359+0000","last_fullsized":"2024-09-18T01:07:37.028359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017852+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:37.017852+0000","last_peered":"2024-09-18T01:07:37.017852+0000","last_clean":"2024-09-18T01:07:37.017852+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:37.017852+0000","last_undegraded":"2024-09-18T01:07:37.017852+0000","last_fullsized":"2024-09-18T01:07:37.017852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'18","reported_seq":214,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541776+0000","last_change":"2024-09-18T01:07:39.033353+0000","last_active":"2024-09-18T01:07:39.541776+0000","last_peered":"2024-09-18T01:07:39.541776+0000","last_clean":"2024-09-18T01:07:39.541776+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:07:39.541776+0000","last_undegraded":"2024-09-18T01:07:39.541776+0000","last_fullsized":"2024-09-18T01:07:39.541776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:56:08.105385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055374899999999997,"stat_sum":{"num_bytes":3686400,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":3632,"num_write":19,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.9","version":"28'6","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-18T01:07:35.143163+0000","last_change":"2024-09-18T01:07:30.977803+0000","last_active":"2024-09-18T01:07:35.143163+0000","last_peered":"2024-09-18T01:07:35.143163+0000","last_clean":"2024-09-18T01:07:35.143163+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:07:35.143163+0000","last_undegraded":"2024-09-18T01:07:35.143163+0000","last_fullsized":"2024-09-18T01:07:35.143163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:07:20.080474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060928,"stat_sum":{"num_bytes":3726593,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"29'14","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.053629+0000","last_change":"2024-09-18T01:07:36.006028+0000","last_active":"2024-09-18T01:07:37.053629+0000","last_peered":"2024-09-18T01:07:37.053629+0000","last_clean":"2024-09-18T01:07:37.053629+0000","last_became_active":"2024-09-18T01:07:28.974981+0000","last_became_peered":"2024-09-18T01:07:28.974981+0000","last_unstale":"2024-09-18T01:07:37.053629+0000","last_undegraded":"2024-09-18T01:07:37.053629+0000","last_fullsized":"2024-09-18T01:07:37.053629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:55:29.455496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040640199999999999,"stat_sum":{"num_bytes":2983076,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":16,"num_write_kb":1657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.b","version":"31'19","reported_seq":383,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-18T01:07:39.541815+0000","last_change":"2024-09-18T01:07:39.033402+0000","last_active":"2024-09-18T01:07:39.541815+0000","last_peered":"2024-09-18T01:07:39.541815+0000","last_clean":"2024-09-18T01:07:39.541815+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:07:39.541815+0000","last_undegraded":"2024-09-18T01:07:39.541815+0000","last_fullsized":"2024-09-18T01:07:39.541815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:04.373023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072990499999999999,"stat_sum":{"num_bytes":9133313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":318,"num_read_kb":5645,"num_write":29,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.c","version":"29'8","reported_seq":233,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.028452+0000","last_change":"2024-09-18T01:07:36.174538+0000","last_active":"2024-09-18T01:07:37.028452+0000","last_peered":"2024-09-18T01:07:37.028452+0000","last_clean":"2024-09-18T01:07:37.028452+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:07:37.028452+0000","last_undegraded":"2024-09-18T01:07:37.028452+0000","last_fullsized":"2024-09-18T01:07:37.028452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:49.390029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097729499999999999,"stat_sum":{"num_bytes":3615160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2939,"num_write":8,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.d","version":"29'11","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018085+0000","last_change":"2024-09-18T01:07:36.178675+0000","last_active":"2024-09-18T01:07:37.018085+0000","last_peered":"2024-09-18T01:07:37.018085+0000","last_clean":"2024-09-18T01:07:37.018085+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:07:37.018085+0000","last_undegraded":"2024-09-18T01:07:37.018085+0000","last_fullsized":"2024-09-18T01:07:37.018085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:55.826275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057819999999999996,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":5,"num_write":11,"num_write_kb":1019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.e","version":"29'6","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018222+0000","last_change":"2024-09-18T01:07:36.178175+0000","last_active":"2024-09-18T01:07:37.018222+0000","last_peered":"2024-09-18T01:07:37.018222+0000","last_clean":"2024-09-18T01:07:37.018222+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:07:37.018222+0000","last_undegraded":"2024-09-18T01:07:37.018222+0000","last_fullsized":"2024-09-18T01:07:37.018222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:15.200273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037671100000000001,"stat_sum":{"num_bytes":1884160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":16,"num_write":7,"num_write_kb":1130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.f","version":"29'3","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.018293+0000","last_change":"2024-09-18T01:07:36.178713+0000","last_active":"2024-09-18T01:07:37.018293+0000","last_peered":"2024-09-18T01:07:37.018293+0000","last_clean":"2024-09-18T01:07:37.018293+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:07:37.018293+0000","last_undegraded":"2024-09-18T01:07:37.018293+0000","last_fullsized":"2024-09-18T01:07:37.018293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:25.935817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062224000000000003,"stat_sum":{"num_bytes":753664,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.10","version":"30'10","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.067551+0000","last_change":"2024-09-18T01:07:36.005973+0000","last_active":"2024-09-18T01:07:37.067551+0000","last_peered":"2024-09-18T01:07:37.067551+0000","last_clean":"2024-09-18T01:07:37.067551+0000","last_became_active":"2024-09-18T01:07:28.976873+0000","last_became_peered":"2024-09-18T01:07:28.976873+0000","last_unstale":"2024-09-18T01:07:37.067551+0000","last_undegraded":"2024-09-18T01:07:37.067551+0000","last_fullsized":"2024-09-18T01:07:37.067551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:14.481178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000360873,"stat_sum":{"num_bytes":702356,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":10,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.11","version":"29'6","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017944+0000","last_change":"2024-09-18T01:07:36.178074+0000","last_active":"2024-09-18T01:07:37.017944+0000","last_peered":"2024-09-18T01:07:37.017944+0000","last_clean":"2024-09-18T01:07:37.017944+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:07:37.017944+0000","last_undegraded":"2024-09-18T01:07:37.017944+0000","last_fullsized":"2024-09-18T01:07:37.017944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:03.904405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034916500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.12","version":"29'17","reported_seq":84,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.024348+0000","last_change":"2024-09-18T01:07:36.014355+0000","last_active":"2024-09-18T01:07:37.024348+0000","last_peered":"2024-09-18T01:07:37.024348+0000","last_clean":"2024-09-18T01:07:37.024348+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:07:37.024348+0000","last_undegraded":"2024-09-18T01:07:37.024348+0000","last_fullsized":"2024-09-18T01:07:37.024348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:04:54.832703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000392972,"stat_sum":{"num_bytes":1812274,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":367,"num_write":17,"num_write_kb":631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.13","version":"29'15","reported_seq":50,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-18T01:07:37.017985+0000","last_change":"2024-09-18T01:07:36.178293+0000","last_active":"2024-09-18T01:07:37.017985+0000","last_peered":"2024-09-18T01:07:37.017985+0000","last_clean":"2024-09-18T01:07:37.017985+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:07:37.017985+0000","last_undegraded":"2024-09-18T01:07:37.017985+0000","last_fullsized":"2024-09-18T01:07:37.017985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:42.884146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049598600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":110380904,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":5021,"num_read_kb":48334,"num_write":1023,"num_write_kb":45350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":75542528,"data_stored":75197448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":732,"ondisk_log_size":732,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37492,"kb_used_data":6276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334348,"statfs":{"total":96636764160,"available":96598372352,"internally_reserved":0,"allocated":6426624,"data_stored":6209367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29520,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342320,"statfs":{"total":96636764160,"available":96606535680,"internally_reserved":0,"allocated":2531328,"data_stored":2323673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":5528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":5660672,"data_stored":5419826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34180,"kb_used_data":7144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337660,"statfs":{"total":96636764160,"available":96601763840,"internally_reserved":0,"allocated":7315456,"data_stored":7089168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40660,"kb_used_data":9460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331180,"statfs":{"total":96636764160,"available":96595128320,"internally_reserved":0,"allocated":9687040,"data_stored":9413434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39540,"kb_used_data":8308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332300,"statfs":{"total":96636764160,"available":96596275200,"internally_reserved":0,"allocated":8507392,"data_stored":8267405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45172,"kb_used_data":13980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326668,"statfs":{"total":96636764160,"available":96590508032,"internally_reserved":0,"allocated":14315520,"data_stored":14088230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55456,"kb_used_data":24264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316384,"statfs":{"total":96636764160,"available":96579977216,"internally_reserved":0,"allocated":24846336,"data_stored":24573927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7593984,"data_stored":7551193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9363456,"data_stored":9287571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7012352,"data_stored":6969503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24489984,"data_stored":24429466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14020608,"data_stored":13974768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5337088,"data_stored":5293963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2211840,"data_stored":2197829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5513216,"data_stored":5493155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:41.237 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:07:41.237 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:07:41.237 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:07:41.238 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:07:41.292 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:41.291+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:41.345 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.345 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.354 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:41.370 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:07:41.398 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:41.397+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:41.458 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.458 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:41.508 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:41.507+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:41.547 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:07:41.556 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:07:41.613 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.613 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.622 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:41.634 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:41.632+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:41.696 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:41.694+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:41.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:07:41.713 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-18T01:07:41.728 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.728 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.740 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:41.797 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:41.797+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:41.836 INFO:teuthology.orchestra.run.smithi120.stdout:256 trunc from 0xe47afd5 to 0xc315f2b 2024-09-18T01:07:41.855 INFO:teuthology.orchestra.run.smithi120.stdout:257 write 0xab757eb thru 0xab81d0d (0xc523 bytes) 2024-09-18T01:07:41.856 INFO:teuthology.orchestra.run.smithi120.stdout:258 read 0xcf3813 thru 0xcf3e28 (0x616 bytes) 2024-09-18T01:07:41.857 INFO:teuthology.orchestra.run.smithi120.stdout:261 punch from 0x1a0921b to 0x1a11fa8, (0x8d8d bytes) 2024-09-18T01:07:41.857 INFO:teuthology.orchestra.run.smithi120.stdout:262 read 0xbb9189d thru 0xbb94a07 (0x316b bytes) 2024-09-18T01:07:41.858 INFO:teuthology.orchestra.run.smithi120.stdout:263 punch from 0x2827cfb to 0x2830ddd, (0x90e2 bytes) 2024-09-18T01:07:41.858 INFO:teuthology.orchestra.run.smithi120.stdout:264 writesame 0xb4722a8 thru 0xb47a092 data_size 0x23(0x7deb bytes) 2024-09-18T01:07:41.859 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:41.859 INFO:teuthology.orchestra.run.smithi120.stdout:266 write 0xab06ea7 thru 0xab1333d (0xc497 bytes) 2024-09-18T01:07:41.861 INFO:teuthology.orchestra.run.smithi120.stdout:267 writesame 0xb1a83c1 thru 0xb1b558d data_size 0x1(0xd1cd bytes) 2024-09-18T01:07:41.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:07:41.873 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:41.873 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:41.882 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:41.885 INFO:teuthology.orchestra.run.smithi120.stdout:269 read 0x2961971 thru 0x2963afa (0x218a bytes) 2024-09-18T01:07:41.885 INFO:teuthology.orchestra.run.smithi120.stdout:270 writesame 0x4f59c7a thru 0x4f60644 data_size 0x35(0x69cb bytes) 2024-09-18T01:07:41.887 INFO:teuthology.orchestra.run.smithi120.stdout:271 write 0xa45aed7 thru 0xa45fff9 (0x5123 bytes) 2024-09-18T01:07:41.889 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xe3eca2a 2024-09-18T01:07:41.892 INFO:teuthology.orchestra.run.smithi120.stdout:272 trunc from 0xc315f2b to 0xe3eca2a 2024-09-18T01:07:41.898 INFO:teuthology.orchestra.run.smithi120.stdout:274 punch from 0x3167877 to 0x317622c, (0xe9b5 bytes) 2024-09-18T01:07:41.898 INFO:teuthology.orchestra.run.smithi120.stdout:275 write 0x97407eb thru 0x974e8ff (0xe115 bytes) 2024-09-18T01:07:41.898 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:41.897+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:41.899 INFO:teuthology.orchestra.run.smithi120.stdout:277 write 0xb33d33 thru 0xb416b4 (0xd982 bytes) 2024-09-18T01:07:41.902 INFO:teuthology.orchestra.run.smithi120.stdout:278 write 0x4c02738 thru 0x4c071c4 (0x4a8d bytes) 2024-09-18T01:07:41.922 INFO:teuthology.orchestra.run.smithi120.stdout:280 trunc from 0xe3eca2a to 0x7a9da4 2024-09-18T01:07:41.941 INFO:teuthology.orchestra.run.smithi120.stdout:281 read 0x19d5d3 thru 0x1a1d46 (0x4774 bytes) 2024-09-18T01:07:41.941 INFO:teuthology.orchestra.run.smithi120.stdout:282 punch from 0x1912d9 to 0x19f3e8, (0xe10f bytes) 2024-09-18T01:07:41.956 INFO:teuthology.orchestra.run.smithi120.stdout:283 trunc from 0x7a9da4 to 0xcb187a4 2024-09-18T01:07:41.964 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:41.964 INFO:teuthology.orchestra.run.smithi120.stdout:286 write 0x60b9f37 thru 0x60bac52 (0xd1c bytes) 2024-09-18T01:07:41.966 INFO:teuthology.orchestra.run.smithi120.stdout:287 punch from 0x9e4a2a0 to 0x9e56967, (0xc6c7 bytes) 2024-09-18T01:07:41.966 INFO:teuthology.orchestra.run.smithi120.stdout:288 read 0xa4062f6 thru 0xa414e16 (0xeb21 bytes) 2024-09-18T01:07:41.967 INFO:teuthology.orchestra.run.smithi120.stdout:289 writesame 0x2d78af6 thru 0x2d85965 data_size 0x12(0xce70 bytes) 2024-09-18T01:07:41.969 INFO:teuthology.orchestra.run.smithi120.stdout:291 read 0x62a6c5 thru 0x63621a (0xbb56 bytes) 2024-09-18T01:07:41.984 INFO:teuthology.orchestra.run.smithi120.stdout:292 trunc from 0xcb187a4 to 0x21e90e3 2024-09-18T01:07:41.999 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:41.998+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:42.002 INFO:teuthology.orchestra.run.smithi120.stdout:293 write 0x2d0cec9 thru 0x2d0daa3 (0xbdb bytes) 2024-09-18T01:07:42.004 INFO:teuthology.orchestra.run.smithi120.stdout:295 writesame 0xa7e3b5 thru 0xa814ed data_size 0x1(0x3139 bytes) 2024-09-18T01:07:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.010 INFO:teuthology.orchestra.run.smithi120.stdout:296 read 0x28502a3 thru 0x28507bf (0x51d bytes) 2024-09-18T01:07:42.010 INFO:teuthology.orchestra.run.smithi120.stdout:297 read 0x4ff8f3 thru 0x505344 (0x5a52 bytes) 2024-09-18T01:07:42.019 INFO:teuthology.orchestra.run.smithi120.stdout:298 trunc from 0x2d0daa4 to 0xa05d0a8 2024-09-18T01:07:42.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:42.025 INFO:teuthology.orchestra.run.smithi120.stdout:299 read 0x95bce5e thru 0x95c18d2 (0x4a75 bytes) 2024-09-18T01:07:42.030 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T01:07:42.032 INFO:teuthology.orchestra.run.smithi120.stdout:301 write 0xb53f7e9 thru 0xb5484eb (0x8d03 bytes) 2024-09-18T01:07:42.042 INFO:teuthology.orchestra.run.smithi120.stdout:302 writesame 0xdfc8c29 thru 0xdfd23a9 data_size 0x1(0x9781 bytes) 2024-09-18T01:07:42.044 INFO:teuthology.orchestra.run.smithi120.stdout:304 punch from 0x12f289e to 0x12fa691, (0x7df3 bytes) 2024-09-18T01:07:42.044 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.050 INFO:teuthology.orchestra.run.smithi120.stdout:306 writesame 0xe709dd6 thru 0xe70b0cc data_size 0x1(0x12f7 bytes) 2024-09-18T01:07:42.051 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.051 INFO:teuthology.orchestra.run.smithi120.stdout:308 write 0x8fd26c6 thru 0x8fd8b61 (0x649c bytes) 2024-09-18T01:07:42.052 INFO:teuthology.orchestra.run.smithi120.stdout:309 punch from 0x440fd47 to 0x441b99c, (0xbc55 bytes) 2024-09-18T01:07:42.052 INFO:teuthology.orchestra.run.smithi120.stdout:310 write 0x3aaef11 thru 0x3ab9669 (0xa759 bytes) 2024-09-18T01:07:42.054 INFO:teuthology.orchestra.run.smithi120.stdout:311 read 0xa5a4dc thru 0xa63ab5 (0x95da bytes) 2024-09-18T01:07:42.054 INFO:teuthology.orchestra.run.smithi120.stdout:313 writesame 0x89f2f11 thru 0x89f54d3 data_size 0x1(0x25c3 bytes) 2024-09-18T01:07:42.059 INFO:teuthology.orchestra.run.smithi120.stdout:314 punch from 0x4014ee7 to 0x401ff7d, (0xb096 bytes) 2024-09-18T01:07:42.059 INFO:teuthology.orchestra.run.smithi120.stdout:316 write 0xe08d9a4 thru 0xe093b91 (0x61ee bytes) 2024-09-18T01:07:42.060 INFO:teuthology.orchestra.run.smithi120.stdout:317 read 0x5f24685 thru 0x5f34435 (0xfdb1 bytes) 2024-09-18T01:07:42.061 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.061 INFO:teuthology.orchestra.run.smithi120.stdout:319 read 0x89fac75 thru 0x89fc694 (0x1a20 bytes) 2024-09-18T01:07:42.061 INFO:teuthology.orchestra.run.smithi120.stdout:320 read 0x47ee6a8 thru 0x47f4875 (0x61ce bytes) 2024-09-18T01:07:42.062 INFO:teuthology.orchestra.run.smithi120.stdout:323 read 0x4500a26 thru 0x4505c3f (0x521a bytes) 2024-09-18T01:07:42.062 INFO:teuthology.orchestra.run.smithi120.stdout:324 writesame 0x887a7e7 thru 0x8880a42 data_size 0xa(0x625c bytes) 2024-09-18T01:07:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:325 punch from 0xcf9385a to 0xcf95cc8, (0x246e bytes) 2024-09-18T01:07:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:327 punch from 0xc420f63 to 0xc428482, (0x751f bytes) 2024-09-18T01:07:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:329 write 0x54308e8 thru 0x543f818 (0xef31 bytes) 2024-09-18T01:07:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:330 read 0x796758b thru 0x797044d (0x8ec3 bytes) 2024-09-18T01:07:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:331 write 0xe961b89 thru 0xe967427 (0x589f bytes) 2024-09-18T01:07:42.080 INFO:teuthology.orchestra.run.smithi120.stdout:333 trunc from 0xe967428 to 0x9b4a42e 2024-09-18T01:07:42.089 INFO:teuthology.orchestra.run.smithi120.stdout:334 read 0x988a94f thru 0x9896e2f (0xc4e1 bytes) 2024-09-18T01:07:42.090 INFO:teuthology.orchestra.run.smithi120.stdout:336 read 0x8a0e8fe thru 0x8a16ba6 (0x82a9 bytes) 2024-09-18T01:07:42.090 INFO:teuthology.orchestra.run.smithi120.stdout:337 read 0x50a58bf thru 0x50b22e5 (0xca27 bytes) 2024-09-18T01:07:42.090 INFO:teuthology.orchestra.run.smithi120.stdout:338 punch from 0x81704cc to 0x817b7d7, (0xb30b bytes) 2024-09-18T01:07:42.091 INFO:teuthology.orchestra.run.smithi120.stdout:339 write 0x128ddb0 thru 0x12944c3 (0x6714 bytes) 2024-09-18T01:07:42.092 INFO:teuthology.orchestra.run.smithi120.stdout:340 read 0x702e1d0 thru 0x703cff9 (0xee2a bytes) 2024-09-18T01:07:42.093 INFO:teuthology.orchestra.run.smithi120.stdout:341 write 0x8eb44f4 thru 0x8ec1697 (0xd1a4 bytes) 2024-09-18T01:07:42.094 INFO:teuthology.orchestra.run.smithi120.stdout:342 read 0x239ddfb thru 0x23aa664 (0xc86a bytes) 2024-09-18T01:07:42.099 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:42.097+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:42.103 INFO:teuthology.orchestra.run.smithi120.stdout:343 write 0xc91b2a4 thru 0xc927d62 (0xcabf bytes) 2024-09-18T01:07:42.106 INFO:teuthology.orchestra.run.smithi120.stdout:345 write 0xb70c8b4 thru 0xb71c647 (0xfd94 bytes) 2024-09-18T01:07:42.107 INFO:teuthology.orchestra.run.smithi120.stdout:348 read 0x6e4db22 thru 0x6e4eaba (0xf99 bytes) 2024-09-18T01:07:42.115 INFO:teuthology.orchestra.run.smithi120.stdout:349 trunc from 0xc927d63 to 0x66d76a0 2024-09-18T01:07:42.128 INFO:teuthology.orchestra.run.smithi120.stdout:351 punch from 0x7a2629 to 0x7b1e23, (0xf7fa bytes) 2024-09-18T01:07:42.134 INFO:teuthology.orchestra.run.smithi120.stdout:352 trunc from 0x66d76a0 to 0xb3be22f 2024-09-18T01:07:42.140 INFO:teuthology.orchestra.run.smithi120.stdout:353 write 0x4e56126 thru 0x4e65db4 (0xfc8f bytes) 2024-09-18T01:07:42.141 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.142 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.142 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.142 INFO:teuthology.orchestra.run.smithi120.stdout:355 writesame 0x554388a thru 0x5550443 data_size 0x59(0xcbba bytes) 2024-09-18T01:07:42.143 INFO:teuthology.orchestra.run.smithi120.stdout:356 punch from 0xa1ea138 to 0xa1eb07f, (0xf47 bytes) 2024-09-18T01:07:42.144 INFO:teuthology.orchestra.run.smithi120.stdout:358 read 0x59f9a52 thru 0x5a0513f (0xb6ee bytes) 2024-09-18T01:07:42.144 INFO:teuthology.orchestra.run.smithi120.stdout:361 writesame 0x3125cbc thru 0x312b154 data_size 0x1(0x5499 bytes) 2024-09-18T01:07:42.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:42.155 INFO:teuthology.orchestra.run.smithi120.stdout:363 write 0x19a17f0 thru 0x19a5dac (0x45bd bytes) 2024-09-18T01:07:42.158 INFO:teuthology.orchestra.run.smithi120.stdout:364 write 0x1612340 thru 0x161a520 (0x81e1 bytes) 2024-09-18T01:07:42.159 INFO:teuthology.orchestra.run.smithi120.stdout:365 write 0xa9ebdb3 thru 0xa9f91ab (0xd3f9 bytes) 2024-09-18T01:07:42.160 INFO:teuthology.orchestra.run.smithi120.stdout:368 read 0x2f60a6c thru 0x2f663ab (0x5940 bytes) 2024-09-18T01:07:42.161 INFO:teuthology.orchestra.run.smithi120.stdout:369 read 0xa3ce5c2 thru 0xa3d22d0 (0x3d0f bytes) 2024-09-18T01:07:42.161 INFO:teuthology.orchestra.run.smithi120.stdout:370 write 0x28cab39 thru 0x28d34ff (0x89c7 bytes) 2024-09-18T01:07:42.162 INFO:teuthology.orchestra.run.smithi120.stdout:372 read 0xdaa0fd thru 0xdb81c7 (0xe0cb bytes) 2024-09-18T01:07:42.163 INFO:teuthology.orchestra.run.smithi120.stdout:373 punch from 0xcdee53 to 0xce9cf7, (0xaea4 bytes) 2024-09-18T01:07:42.176 INFO:teuthology.orchestra.run.smithi120.stdout:374 trunc from 0xb3be22f to 0x88385a 2024-09-18T01:07:42.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T01:07:42.196 INFO:teuthology.orchestra.run.smithi120.stdout:375 trunc from 0x88385a to 0xa374531 2024-09-18T01:07:42.211 INFO:teuthology.orchestra.run.smithi120.stdout:376 trunc from 0xa374531 to 0x3be6262 2024-09-18T01:07:42.211 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:42.209+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:42.222 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.222 INFO:teuthology.orchestra.run.smithi120.stdout:379 punch from 0x2af21f3 to 0x2b00f9d, (0xedaa bytes) 2024-09-18T01:07:42.240 INFO:teuthology.orchestra.run.smithi120.stdout:380 write 0xbb69112 thru 0xbb7054c (0x743b bytes) 2024-09-18T01:07:42.244 INFO:teuthology.orchestra.run.smithi120.stdout:382 punch from 0x4cab81e to 0x4cb7d34, (0xc516 bytes) 2024-09-18T01:07:42.244 INFO:teuthology.orchestra.run.smithi120.stdout:385 punch from 0x9a54de2 to 0x9a5c8c4, (0x7ae2 bytes) 2024-09-18T01:07:42.244 INFO:teuthology.orchestra.run.smithi120.stdout:386 writesame 0x24512a0 thru 0x2453c51 data_size 0x9(0x29b2 bytes) 2024-09-18T01:07:42.245 INFO:teuthology.orchestra.run.smithi120.stdout:387 read 0x54989d9 thru 0x549aa08 (0x2030 bytes) 2024-09-18T01:07:42.246 INFO:teuthology.orchestra.run.smithi120.stdout:388 read 0x4b2c87e thru 0x4b2e8af (0x2032 bytes) 2024-09-18T01:07:42.246 INFO:teuthology.orchestra.run.smithi120.stdout:389 punch from 0x2a08a10 to 0x2a0f54c, (0x6b3c bytes) 2024-09-18T01:07:42.246 INFO:teuthology.orchestra.run.smithi120.stdout:390 read 0x80ec754 thru 0x80f923b (0xcae8 bytes) 2024-09-18T01:07:42.247 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.247 INFO:teuthology.orchestra.run.smithi120.stdout:392 read 0x7c2e5bf thru 0x7c3a4bd (0xbeff bytes) 2024-09-18T01:07:42.247 INFO:teuthology.orchestra.run.smithi120.stdout:394 write 0x481086c thru 0x48156d6 (0x4e6b bytes) 2024-09-18T01:07:42.251 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.251 INFO:teuthology.orchestra.run.smithi120.stdout:397 write 0x3b98dbb thru 0x3ba8787 (0xf9cd bytes) 2024-09-18T01:07:42.252 INFO:teuthology.orchestra.run.smithi120.stdout:398 punch from 0xaffe992 to 0xb00704c, (0x86ba bytes) 2024-09-18T01:07:42.252 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.260 INFO:teuthology.orchestra.run.smithi120.stdout:400 write 0xd2d1e2d thru 0xd2d99ba (0x7b8e bytes) 2024-09-18T01:07:42.261 INFO:teuthology.orchestra.run.smithi120.stdout:401 read 0x2a7e254 thru 0x2a84494 (0x6241 bytes) 2024-09-18T01:07:42.262 INFO:teuthology.orchestra.run.smithi120.stdout:402 read 0xa191057 thru 0xa193c38 (0x2be2 bytes) 2024-09-18T01:07:42.262 INFO:teuthology.orchestra.run.smithi120.stdout:403 read 0x5db23c9 thru 0x5db2534 (0x16c bytes) 2024-09-18T01:07:42.262 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.262 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.262 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xe75a9b2 2024-09-18T01:07:42.264 INFO:teuthology.orchestra.run.smithi120.stdout:408 trunc from 0xd2d99bb to 0xe75a9b2 2024-09-18T01:07:42.269 INFO:teuthology.orchestra.run.smithi120.stdout:409 write 0x2765660 thru 0x2767efe (0x289f bytes) 2024-09-18T01:07:42.271 INFO:teuthology.orchestra.run.smithi120.stdout:410 write 0x6d23e96 thru 0x6d2daa4 (0x9c0f bytes) 2024-09-18T01:07:42.273 INFO:teuthology.orchestra.run.smithi120.stdout:411 punch from 0x96181f5 to 0x961f9a2, (0x77ad bytes) 2024-09-18T01:07:42.273 INFO:teuthology.orchestra.run.smithi120.stdout:413 write 0x68a3f45 thru 0x68a647d (0x2539 bytes) 2024-09-18T01:07:42.275 INFO:teuthology.orchestra.run.smithi120.stdout:414 trunc from 0xe75a9b2 to 0xde9d444 2024-09-18T01:07:42.285 INFO:teuthology.orchestra.run.smithi120.stdout:415 writesame 0xd0cc380 thru 0xd0dae71 data_size 0x3d(0xeaf2 bytes) 2024-09-18T01:07:42.286 INFO:teuthology.orchestra.run.smithi120.stdout:416 writesame 0x3f60d38 thru 0x3f6b707 data_size 0x13(0xa9d0 bytes) 2024-09-18T01:07:42.288 INFO:teuthology.orchestra.run.smithi120.stdout:417 punch from 0x317d310 to 0x317e00c, (0xcfc bytes) 2024-09-18T01:07:42.288 INFO:teuthology.orchestra.run.smithi120.stdout:418 write 0x33b6e12 thru 0x33c14f6 (0xa6e5 bytes) 2024-09-18T01:07:42.290 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.290 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.290 INFO:teuthology.orchestra.run.smithi120.stdout:419 punch from 0xd1e776f to 0xd1e7e52, (0x6e3 bytes) 2024-09-18T01:07:42.298 INFO:teuthology.orchestra.run.smithi120.stdout:421 trunc from 0xde9d444 to 0x70c920e 2024-09-18T01:07:42.300 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:42.299+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:42.304 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:42.313 INFO:teuthology.orchestra.run.smithi120.stdout:422 read 0x50e7ecb thru 0x50f025b (0x8391 bytes) 2024-09-18T01:07:42.317 INFO:teuthology.orchestra.run.smithi120.stdout:423 trunc from 0x70c920e to 0xa9eda06 2024-09-18T01:07:42.323 INFO:teuthology.orchestra.run.smithi120.stdout:426 writesame 0x5e4e58 thru 0x5e9715 data_size 0x1(0x48be bytes) 2024-09-18T01:07:42.332 INFO:teuthology.orchestra.run.smithi120.stdout:427 writesame 0x23bf87e thru 0x23ccdde data_size 0x73(0xd561 bytes) 2024-09-18T01:07:42.334 INFO:teuthology.orchestra.run.smithi120.stdout:428 write 0x6518457 thru 0x651dfd8 (0x5b82 bytes) 2024-09-18T01:07:42.336 INFO:teuthology.orchestra.run.smithi120.stdout:429 writesame 0x6b7e1dc thru 0x6b86dd9 data_size 0x63(0x8bfe bytes) 2024-09-18T01:07:42.346 INFO:teuthology.orchestra.run.smithi120.stdout:430 writesame 0xb26da4a thru 0xb278b23 data_size 0x1(0xb0da bytes) 2024-09-18T01:07:42.347 INFO:teuthology.orchestra.run.smithi120.stdout:432 write 0x1226e5f thru 0x122fe67 (0x9009 bytes) 2024-09-18T01:07:42.355 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T01:07:42.359 INFO:teuthology.orchestra.run.smithi120.stdout:433 write 0xcd0b33c thru 0xcd18b25 (0xd7ea bytes) 2024-09-18T01:07:42.361 INFO:teuthology.orchestra.run.smithi120.stdout:434 write 0x8328f34 thru 0x8338392 (0xf45f bytes) 2024-09-18T01:07:42.363 INFO:teuthology.orchestra.run.smithi120.stdout:437 write 0x2e1b0e0 thru 0x2e263e6 (0xb307 bytes) 2024-09-18T01:07:42.365 INFO:teuthology.orchestra.run.smithi120.stdout:438 writesame 0x903abeb thru 0x9047580 data_size 0x5e(0xc996 bytes) 2024-09-18T01:07:42.366 INFO:teuthology.orchestra.run.smithi120.stdout:439 write 0xa545bf thru 0xa5796f (0x33b1 bytes) 2024-09-18T01:07:42.369 INFO:teuthology.orchestra.run.smithi120.stdout:440 write 0x100ba2a thru 0x1018aab (0xd082 bytes) 2024-09-18T01:07:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:443 write 0x1842292 thru 0x1843330 (0x109f bytes) 2024-09-18T01:07:42.372 INFO:teuthology.orchestra.run.smithi120.stdout:444 clone 4 order 18 su 131072 sc 6 2024-09-18T01:07:42.401 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:42.399+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:42.423 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.423 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.433 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:42.537 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:42.535+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:42.545 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:07:42.587 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.587 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:42.602 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:42.601+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:42.703 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:42.702+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:42.739 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.739 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.749 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:42.805 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:42.804+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:42.851 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.851 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.860 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:42.905 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:42.904+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:42.963 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:42.963 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:42.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:43.006 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:43.005+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:43.043 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:43.076 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:43.076 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:43.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:43.127 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:43.125+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:43.165 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:43.166 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:43.182 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:43.183 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:43.199 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:43.199 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:43.207 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:43.206+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:43.215 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:43.215 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:43.232 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:43.233 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:43.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:43.249 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:43.266 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:43.266 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:43.283 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:43.283 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:43.335 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:43.333+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:43.354 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8294 3.7 3:e3e34f1e:::rbd_data.11762d3aac1.0000000000000000:head [write 0~524288 [fadvise_sequential+fadvise_nocache],zero 524288~524288,write 1048576~524288 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~1048576 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7678435179999998, 2024-09-18T01:07:43.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027761263000000001, 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8294 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586908+0000", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022325259999999999 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586922+0000", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4061e-05 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586931+0000", 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2089999999999994e-06 2024-09-18T01:07:43.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587661+0000", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00073032299999999995 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587811+0000", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000150047 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.588205+0000", 2024-09-18T01:07:43.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039331099999999998 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610502+0000", 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022297437 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612252+0000", 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017498940000000001 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612391+0000", 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013893099999999999 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612436+0000", 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5093999999999999e-05 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8316 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 1048576~163840 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7654718900000002, 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012174538, 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.362 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8316 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:43.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587047+0000", 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0400000000000002e-07 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587049+0000", 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.728e-06 2024-09-18T01:07:43.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587055+0000", 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9909999999999999e-06 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587060+0000", 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5360000000000003e-06 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595805+0000", 2024-09-18T01:07:43.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087449510000000008 2024-09-18T01:07:43.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595925+0000", 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012037200000000001 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599221+0000", 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032955559999999998 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8324 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 1572864~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:43.368 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7651659349999997, 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019081754999999999, 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8324 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587353+0000", 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2399999999999998e-07 2024-09-18T01:07:43.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587356+0000", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9450000000000002e-06 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587363+0000", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.452e-06 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587369+0000", 2024-09-18T01:07:43.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5659999999999997e-06 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599368+0000", 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011999604000000001 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599490+0000", 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121357 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606434+0000", 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006944207 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8341 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.765089438, 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02321252, 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8341 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.374 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587435+0000", 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9610000000000002e-06 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587442+0000", 2024-09-18T01:07:43.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7000000000000002e-06 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587447+0000", 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606471+0000", 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019024926000000001 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606525+0000", 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3958000000000001e-05 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610641+0000", 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041158519999999997 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.378 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8348 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7586332799999997, 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013699756, 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8348 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593887+0000", 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2400000000000002e-06 2024-09-18T01:07:43.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593902+0000", 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4421e-05 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593922+0000", 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0020000000000001e-05 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593929+0000", 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7759999999999997e-06 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594130+0000", 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020075599999999999 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594241+0000", 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001111 2024-09-18T01:07:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594643+0000", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00040180700000000001 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600371+0000", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057278600000000004 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607482+0000", 2024-09-18T01:07:43.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071108730000000002 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607536+0000", 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4230000000000001e-05 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607585+0000", 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8673e-05 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.384 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8350 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7582338630000001, 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017945170999999999, 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.385 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8350 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594285+0000", 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000004e-07 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594290+0000", 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1710000000000001e-06 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594301+0000", 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0499000000000001e-05 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594309+0000", 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9650000000000008e-06 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610528+0000", 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016218713999999999 2024-09-18T01:07:43.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610576+0000", 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8368999999999997e-05 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612230+0000", 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016536280000000001 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8390 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7564990979999999, 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01144247, 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8390 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596020+0000", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596022+0000", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8310000000000001e-06 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596029+0000", 2024-09-18T01:07:43.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6549999999999998e-06 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596044+0000", 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4574e-05 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600514+0000", 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044701209999999996 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600615+0000", 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100936 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607462+0000", 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068468139999999997 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8399 3.2 3:44ca0a04:::rbd_data.11762d3aac1.0000000000000006:head [write 0~3670016 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache],zero 5767168~524288,zero 7602176~262144,zero 15990784~218866,write 16209650~22755 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.738806329, 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029285142, 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8399 2024-09-18T01:07:43.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613714+0000", 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9180000000000001e-06 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615140+0000", 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014266820000000001 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615157+0000", 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6330000000000001e-05 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615166+0000", 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615300+0000", 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013375400000000001 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615497+0000", 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019682799999999999 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.616109+0000", 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061267699999999995 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642827+0000", 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026717366999999999 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642859+0000", 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2777999999999997e-05 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642961+0000", 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101137 2024-09-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642997+0000", 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6449000000000003e-05 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8447 3.2 3:452cb511:::rbd_data.1176ec0d676b.0000000000000058:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:43.401 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7371346440000002, 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013050262, 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8447 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615386+0000", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615392+0000", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.037e-06 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615396+0000", 2024-09-18T01:07:43.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628305+0000", 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012908974 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628434+0000", 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012840200000000001 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8457 3.2 3:46e71cef:::rbd_data.1176ec0d676b.000000000000006a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.737093164, 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013141890999999999, 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.406 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8457 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615426+0000", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615428+0000", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.046e-06 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615433+0000", 2024-09-18T01:07:43.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615437+0000", 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9380000000000002e-06 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628468+0000", 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013030570999999999 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628567+0000", 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9035000000000003e-05 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:8397 3.5 e34/23 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head v 34'8, mlcod=31'6)", 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7335715619999998, 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030584585000000001, 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:43.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627937+0000", 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089904949999999994 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627956+0000", 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8984999999999999e-05 2024-09-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627964+0000", 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5129999999999999e-06 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628036+0000", 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2004999999999999e-05 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628066+0000", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9569000000000001e-05 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649485+0000", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021419779 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649531+0000", 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5948999999999997e-05 2024-09-18T01:07:43.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8785 3.2 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head [write 0~1146641 [fadvise_sequential+fadvise_nocache],zero 1146641~426223,write 1572864~2621440 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.727070576, 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028310961999999999, 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8785 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625449+0000", 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.773e-06 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627661+0000", 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022114980000000001 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627688+0000", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6988e-05 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627697+0000", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8710000000000003e-06 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629596+0000", 2024-09-18T01:07:43.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001899155 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630212+0000", 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061557099999999998 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630882+0000", 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00067058700000000005 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649558+0000", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018675605000000001 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653628+0000", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004070097 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653707+0000", 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9645000000000001e-05 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653759+0000", 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1171999999999997e-05 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8787 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.720352025, 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021599920000000002, 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8787 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632169+0000", 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:07:43.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632181+0000", 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1564e-05 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632203+0000", 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632227+0000", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3484000000000001e-05 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632310+0000", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2891000000000003e-05 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632426+0000", 2024-09-18T01:07:43.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116288 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635044+0000", 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026176419999999999 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645349+0000", 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010305527 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653684+0000", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083345120000000005 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653725+0000", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1119999999999999e-05 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653766+0000", 2024-09-18T01:07:43.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1287e-05 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8824 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7187599000000002, 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011546005, 2024-09-18T01:07:43.428 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8824 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633759+0000", 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633763+0000", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1189999999999999e-06 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633770+0000", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9129999999999997e-06 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633775+0000", 2024-09-18T01:07:43.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645111+0000", 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011336061999999999 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645226+0000", 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114784 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645304+0000", 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8510999999999994e-05 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8960 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:43.433 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7178004229999999, 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.014368275, 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.434 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8960 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1200000000000003e-07 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634720+0000", 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:07:43.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634725+0000", 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634735+0000", 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0176999999999999e-05 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643266+0000", 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085303800000000006 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643358+0000", 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1699000000000004e-05 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649086+0000", 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057285180000000002 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8965 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 786432~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7170189389999999, 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015273905000000001, 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8965 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:43.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635500+0000", 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-06 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635505+0000", 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635516+0000", 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1089e-05 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635529+0000", 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2754e-05 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649126+0000", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013596964 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649214+0000", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7905000000000006e-05 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.650773+0000", 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001559305 2024-09-18T01:07:43.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8973 3.1e 3:7e62f05a:::rbd_data.1176ec0d676b.0000000000000012:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7169769189999999, 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01383679, 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.444 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8973 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635542+0000", 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9599999999999999e-07 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635544+0000", 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4109999999999999e-06 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635551+0000", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8109999999999997e-06 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635556+0000", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8550000000000001e-06 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646171+0000", 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010615342 2024-09-18T01:07:43.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646218+0000", 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7259000000000003e-05 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649378+0000", 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031596160000000001 2024-09-18T01:07:43.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8982 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1310720~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7169555490000001, 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015554682, 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8982 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635565+0000", 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8470000000000001e-06 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635571+0000", 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7219999999999996e-06 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635575+0000", 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649410+0000", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013835055000000001 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649481+0000", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0820000000000003e-05 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.651117+0000", 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016363770000000001 2024-09-18T01:07:43.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9166 3.18 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.6973735720000001, 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028895934000000002, 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9166 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655147+0000", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9879999999999999e-06 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656773+0000", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00162633 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656788+0000", 2024-09-18T01:07:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4547e-05 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656795+0000", 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2989999999999999e-06 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656857+0000", 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2203000000000001e-05 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656994+0000", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013699599999999999 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657438+0000", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044429499999999999 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.682244+0000", 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024805437999999999 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683944+0000", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001699926 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683993+0000", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9447999999999998e-05 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.684041+0000", 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7463999999999998e-05 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9173 3.19 e34/23 3:9b92acfd:::rbd_data.11762d3aac1.000000000000000e:head v 34'19, mlcod=31'17)", 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.6852217820000002, 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016520503999999998, 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667297+0000", 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3300000000000001e-07 2024-09-18T01:07:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668283+0000", 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098571899999999992 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668305+0000", 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1665999999999999e-05 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668317+0000", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2013e-05 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668402+0000", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4970999999999997e-05 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668429+0000", 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7416e-05 2024-09-18T01:07:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683769+0000", 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015339999999999999 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683817+0000", 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7886000000000002e-05 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:43.465 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:43.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:43.467 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:43.465+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:43.567 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:43.565+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:43.595 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:10053 3.9 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head [write 0~262144 [fadvise_sequential+fadvise_nocache] in=262144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:43.592758+0000", 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.0019230600000000001, 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019559099999999999, 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10053 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.592758+0000", 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.592758+0000", 2024-09-18T01:07:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.592760+0000", 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0150000000000002e-06 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593165+0000", 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040520699999999998 2024-09-18T01:07:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593184+0000", 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8677000000000001e-05 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593193+0000", 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.738e-06 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593257+0000", 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4397e-05 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593396+0000", 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013897700000000001 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.593541+0000", 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000145165 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.594630+0000", 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00108888 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-18T01:07:43.601 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:43.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:43.667 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:43.666+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:43.740 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:43.740 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:43.740 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:43.740 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:43.741 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:43.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:43.768 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:43.767+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:43.870 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:43.869+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:43.920 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:43.920 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:43.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.943320545000001, 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079564049999999997, 2024-09-18T01:07:43.921 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9 2024-09-18T01:07:43.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974229+0000", 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974232+0000", 2024-09-18T01:07:43.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974246+0000", 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4326e-05 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974271+0000", 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.525e-05 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974286+0000", 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4653e-05 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974374+0000", 2024-09-18T01:07:43.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8067999999999998e-05 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.974382+0000", 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5870000000000004e-06 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.976986+0000", 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026044570000000001 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977054+0000", 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7533000000000005e-05 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.977201+0000", 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014775300000000001 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.978421+0000", 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012200780000000001 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982094+0000", 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036729810000000001 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982145+0000", 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0970999999999997e-05 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:30.982185+0000", 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9878e-05 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2197 3.14 3:2e019870:::rbd_data.11769efc726f.0000000000000042:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7412253030000002, 2024-09-18T01:07:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076807569999999999, 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2197 2024-09-18T01:07:43.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176324+0000", 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176325+0000", 2024-09-18T01:07:43.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3500000000000005e-07 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176902+0000", 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057736500000000004 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176915+0000", 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2201000000000001e-05 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176920+0000", 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.164e-06 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.176959+0000", 2024-09-18T01:07:43.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8955000000000002e-05 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177051+0000", 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2440000000000003e-05 2024-09-18T01:07:43.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177147+0000", 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5685000000000003e-05 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.181258+0000", 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041109440000000001 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183932+0000", 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002674184 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.183971+0000", 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8621000000000002e-05 2024-09-18T01:07:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.184005+0000", 2024-09-18T01:07:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4263e-05 2024-09-18T01:07:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:2199 3.1d e28/23 3:bd875124:::rbd_data.11769efc726f.000000000000003f:head v 28'4, mlcod=26'2)", 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7396876460000001, 2024-09-18T01:07:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085995269999999992, 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177862+0000", 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177863+0000", 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177946+0000", 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3165999999999996e-05 2024-09-18T01:07:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177956+0000", 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0617e-05 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.177963+0000", 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5470000000000002e-06 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178020+0000", 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6954999999999999e-05 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.178039+0000", 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.965e-05 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186422+0000", 2024-09-18T01:07:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083826030000000006 2024-09-18T01:07:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.186461+0000", 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9169999999999999e-05 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3111 3.3 3:c5270ba5:::rbd_data.11769efc726f.0000000000000045:head [write 327680~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.7507998479999998, 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064345019999999999, 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3111 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166750+0000", 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166768+0000", 2024-09-18T01:07:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8447000000000001e-05 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166774+0000", 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5620000000000003e-06 2024-09-18T01:07:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166777+0000", 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1599999999999998e-06 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166827+0000", 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0037999999999998e-05 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.166926+0000", 2024-09-18T01:07:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8954000000000002e-05 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.167073+0000", 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147317 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.169081+0000", 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002007708 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173084+0000", 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040024220000000003 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173139+0000", 2024-09-18T01:07:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5689999999999997e-05 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.173184+0000", 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4821000000000003e-05 2024-09-18T01:07:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3232 3.9 3:91310cee:::rbd_data.11769efc726f.00000000000000bf:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6179718159999998, 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064668900000000003, 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3232 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299578+0000", 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6899999999999998e-07 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299582+0000", 2024-09-18T01:07:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299590+0000", 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000001e-06 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299595+0000", 2024-09-18T01:07:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-18T01:07:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300116+0000", 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052083200000000004 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300194+0000", 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7818000000000006e-05 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300294+0000", 2024-09-18T01:07:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100138 2024-09-18T01:07:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.302169+0000", 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018753960000000001 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.305978+0000", 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038083610000000001 2024-09-18T01:07:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306017+0000", 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9487000000000003e-05 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306044+0000", 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7367000000000001e-05 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3237 3.1b e29/23 3:de903dd5:::rbd_data.11769efc726f.00000000000000d6:head v 29'49, mlcod=29'47)", 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6172728090000001, 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057268120000000004, 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300277+0000", 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300284+0000", 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1130000000000003e-06 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300294+0000", 2024-09-18T01:07:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300305+0000", 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.047e-05 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300353+0000", 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8689000000000001e-05 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300375+0000", 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1175999999999999e-05 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.305975+0000", 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005599924 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.306003+0000", 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3261 3.1b e29/23 3:dc449506:::rbd_object_map.11769efc726f:head v 29'56, mlcod=29'52)", 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.5948475020000004, 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067026209999999998, 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322702+0000", 2024-09-18T01:07:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322703+0000", 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000003e-07 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322708+0000", 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7169999999999996e-06 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322719+0000", 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1184e-05 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322726+0000", 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1310000000000003e-06 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322768+0000", 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2706000000000001e-05 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.322787+0000", 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8428999999999999e-05 2024-09-18T01:07:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329369+0000", 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065823870000000003 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329404+0000", 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5369e-05 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3263 3.1d e29/23 3:bf6d2803:::rbd_data.11769efc726f.00000000000000b1:head v 29'16, mlcod=29'14)", 2024-09-18T01:07:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.5939521230000002, 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058123380000000002, 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323597+0000", 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323598+0000", 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323613+0000", 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5325e-05 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323622+0000", 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999993e-06 2024-09-18T01:07:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323627+0000", 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4020000000000004e-06 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323663+0000", 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6356999999999998e-05 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323675+0000", 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2254e-05 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329386+0000", 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005710283 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329410+0000", 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4051000000000001e-05 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3264 3.1b e29/23 3:dfb219e3:::rbd_data.11769efc726f.00000000000000b2:head v 29'57, mlcod=29'54)", 2024-09-18T01:07:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.5938568860000002, 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057289050000000003, 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323692+0000", 2024-09-18T01:07:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323693+0000", 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323704+0000", 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0773e-05 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323710+0000", 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8300000000000001e-06 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323713+0000", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3969999999999998e-06 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323747+0000", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.434e-05 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.323763+0000", 2024-09-18T01:07:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.59e-05 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329407+0000", 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005643599 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.329421+0000", 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4574e-05 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.326088999, 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:07:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.3177573269999998, 2024-09-18T01:07:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:07:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.3144164730000001, 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:07:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2869616439999998, 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:07:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:07:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8910 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.635851+0000", 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2816986099999998, 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006365936, 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8910 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635851+0000", 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635851+0000", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635851+0000", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635854+0000", 2024-09-18T01:07:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635860+0000", 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635864+0000", 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2490000000000003e-06 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.641032+0000", 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0051679669999999999 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.641134+0000", 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102351 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.642217+0000", 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010825240000000001 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8964 3.1a 3:5d7f1f78:::rbd_data.1176ec0d676b.0000000000000037:7 [sparse-read 524288~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.635948+0000", 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2816010840000001, 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064914899999999999, 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8964 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635948+0000", 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635948+0000", 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635949+0000", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4799999999999999e-07 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635951+0000", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635956+0000", 2024-09-18T01:07:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8669999999999999e-06 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635961+0000", 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439999999999996e-06 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.642256+0000", 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006295536 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.642335+0000", 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9246999999999995e-05 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.642440+0000", 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104431 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8971 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:7 [sparse-read 0~115227 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.635984+0000", 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.281565413, 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061029500000000002, 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8971 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635984+0000", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635984+0000", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635984+0000", 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635986+0000", 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.99e-06 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635992+0000", 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.635997+0000", 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3490000000000002e-06 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640224+0000", 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042274399999999998 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640321+0000", 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7411000000000006e-05 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.642087+0000", 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001765473 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9122 3.1a 3:597bcf6d:::rbd_data.1176ec0d676b.000000000000000e:7 [sparse-read 1310720~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.646638+0000", 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2709114349999999, 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064989510000000002, 2024-09-18T01:07:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9122 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646638+0000", 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646638+0000", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646639+0000", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8500000000000001e-07 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646650+0000", 2024-09-18T01:07:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1039000000000001e-05 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646659+0000", 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646665+0000", 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646922+0000", 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000257268 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646968+0000", 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5784000000000001e-05 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653137+0000", 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061691280000000003 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9131 3.3 3:c0e3c397:::rbd_data.1176ec0d676b.000000000000001d:7 [sparse-read 1572864~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.646748+0000", 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2708012389999999, 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005858495, 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9131 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646748+0000", 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646748+0000", 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646749+0000", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7100000000000001e-07 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646755+0000", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8730000000000004e-06 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646764+0000", 2024-09-18T01:07:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2259999999999994e-06 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646770+0000", 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1789999999999996e-06 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646874+0000", 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010395099999999999 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646959+0000", 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4503000000000001e-05 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.652607+0000", 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056480920000000004 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9138 3.1a 3:5d7f1f78:::rbd_data.1176ec0d676b.0000000000000037:7 [sparse-read 1835008~48183 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.646844+0000", 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2707052239999999, 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0068639859999999999, 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9138 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646844+0000", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646844+0000", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646845+0000", 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4000000000000001e-07 2024-09-18T01:07:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646849+0000", 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6780000000000002e-06 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646858+0000", 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4250000000000004e-06 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.646874+0000", 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5593000000000001e-05 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653164+0000", 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006289996 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653243+0000", 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9845999999999994e-05 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653708+0000", 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046470799999999998 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.252996327, 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:07:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:07:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:07:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.059 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.060 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:44.061 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:44.059+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:44.162 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:44.160+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:44.205 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:44.205 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:44.205 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:44.205 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.214 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:44.261 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:44.259+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:44.356 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:44.356 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:44.356 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:44.356 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:44.356 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.362 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:44.362+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:44.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:44.464 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:44.463+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.484962865, 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.510 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:07:44.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4849207139999994, 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:44.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4848872859999993, 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4848497859999998, 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:07:44.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:07:44.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4848282089999998, 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4848090690000006, 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:07:44.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4847886779999993, 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:07:44.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:07:44.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4847592780000003, 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4847304880000003, 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.548 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:44.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4847097930000004, 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:07:44.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4846903089999994, 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:07:44.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:07:44.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4846563980000003, 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4846203990000006, 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:07:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:07:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4001724269999993, 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:07:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:07:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4001622180000002, 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:07:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.899760992, 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:07:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:07:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8842748230000002, 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:07:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8763841029999999, 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:07:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:07:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8503151, 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8451463079999999, 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:07:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:07:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:07:44.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:44.609 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:07:44.611 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:44.612 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:44.609+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:44.691 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-18T01:07:44.711 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:44.710+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:44.765 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:44.765 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:44.765 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:44.765 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.775 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:44.813 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:44.811+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:44.914 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:44.913+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:44.918 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:44.918 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:44.918 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:44.918 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:44.918 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:44.927 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:44.983 INFO:teuthology.orchestra.run.smithi120.stdout:448 trunc from 0xcd18b26 to 0xe5a1ec3 2024-09-18T01:07:44.991 INFO:teuthology.orchestra.run.smithi120.stdout:450 writesame 0x4655625 thru 0x4661d92 data_size 0x86(0xc76e bytes) 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:454 read 0x3b6e766 thru 0x3b7e0f2 (0xf98d bytes) 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:457 punch from 0x4444f08 to 0x4450ac8, (0xbbc0 bytes) 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:461 read 0x3abcb8 thru 0x3bb86b (0xfbb4 bytes) 2024-09-18T01:07:45.007 INFO:teuthology.orchestra.run.smithi120.stdout:462 trunc from 0xe5a1ec3 to 0x6017da7 2024-09-18T01:07:45.014 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:45.013+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:45.053 INFO:teuthology.orchestra.run.smithi120.stdout:463 read 0x306583a thru 0x3068986 (0x314d bytes) 2024-09-18T01:07:45.053 INFO:teuthology.orchestra.run.smithi120.stdout:465 read 0x2969f04 thru 0x296b998 (0x1a95 bytes) 2024-09-18T01:07:45.053 INFO:teuthology.orchestra.run.smithi120.stdout:466 write 0x36c7e54 thru 0x36ce1b3 (0x6360 bytes) 2024-09-18T01:07:45.072 INFO:teuthology.orchestra.run.smithi120.stdout:467 write 0xda72c16 thru 0xda786aa (0x5a95 bytes) 2024-09-18T01:07:45.085 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:45.085 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:45.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:45.085 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:45.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.000206044, 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:07:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:07:45.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:07:45.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:07:45.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:45.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:07:45.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:07:45.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1682 3.a 3:51147147:::rbd_data.11769efc726f.0000000000000027:head [write 524288~410788 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.136580+0000", 2024-09-18T01:07:45.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.9451620750000007, 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035433249999999999, 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1682 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136580+0000", 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136580+0000", 2024-09-18T01:07:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136582+0000", 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:07:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136947+0000", 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036531900000000002 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136987+0000", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9509999999999999e-05 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.136999+0000", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.243e-05 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.137058+0000", 2024-09-18T01:07:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8217000000000002e-05 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.137210+0000", 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000152378 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.137322+0000", 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011232599999999999 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.139472+0000", 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021498519999999998 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.140032+0000", 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00056022799999999996 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.140080+0000", 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7253999999999999e-05 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.140123+0000", 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3835999999999998e-05 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1815 3.a 3:5642d443:::rbd_data.11769efc726f.000000000000002f:head [write 0~180224 [fadvise_sequential+fadvise_nocache],write 524288~245760 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.9340662959999992, 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053689159999999996, 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.108 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1815 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147678+0000", 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148125+0000", 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000447157 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148138+0000", 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3664e-05 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148146+0000", 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.61e-06 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148218+0000", 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1854999999999996e-05 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148347+0000", 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012892599999999999 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148516+0000", 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000168925 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152752+0000", 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042366449999999998 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152932+0000", 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017960200000000001 2024-09-18T01:07:45.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152993+0000", 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1162999999999997e-05 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.153045+0000", 2024-09-18T01:07:45.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1625000000000001e-05 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:1819 3.1e e28/23 3:7ef3d4e5:::rbd_data.11769efc726f.0000000000000038:head v 28'6, mlcod=26'3)", 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.148371+0000", 2024-09-18T01:07:45.116 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.9333715609999995, 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041309329999999998, 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148371+0000", 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148371+0000", 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148371+0000", 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8800000000000002e-07 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148728+0000", 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035661299999999998 2024-09-18T01:07:45.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148741+0000", 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3193999999999999e-05 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148748+0000", 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9079999999999998e-06 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148806+0000", 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8125000000000003e-05 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148839+0000", 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2620999999999998e-05 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152458+0000", 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036194420000000001 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152502+0000", 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3341999999999999e-05 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2636 3.a 3:5581bfb7:::rbd_data.11769efc726f.0000000000000093:head [write 0~245760 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.8410418279999998, 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043618010000000002, 2024-09-18T01:07:45.122 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2636 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240702+0000", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240918+0000", 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021542999999999999 2024-09-18T01:07:45.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240934+0000", 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6192999999999999e-05 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240941+0000", 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1940000000000001e-06 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241013+0000", 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1555000000000002e-05 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241121+0000", 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107772 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241263+0000", 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141953 2024-09-18T01:07:45.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.242614+0000", 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013513710000000001 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.244954+0000", 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002340066 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245013+0000", 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9358999999999999e-05 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245062+0000", 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8869999999999998e-05 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2689 3.a 3:5042a8b3:::rbd_data.11769efc726f.00000000000000a9:head [write 524288~147456 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.245959+0000", 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.8357828250000008, 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039352220000000004, 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2689 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245959+0000", 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245959+0000", 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245966+0000", 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246092+0000", 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000126024 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246104+0000", 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2344000000000001e-05 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246112+0000", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4429999999999997e-06 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246172+0000", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.003e-05 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246316+0000", 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014426599999999999 2024-09-18T01:07:45.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.246458+0000", 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141822 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.247912+0000", 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014544950000000001 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.249823+0000", 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019109839999999999 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.249867+0000", 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3498000000000003e-05 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.249895+0000", 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7682e-05 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3022 3.1e e28/23 3:7bb42000:::rbd_data.11769efc726f.00000000000000e9:head v 28'8, mlcod=28'6)", 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7832120879999991, 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045301040000000001, 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298531+0000", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.212e-06 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298648+0000", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116434 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298665+0000", 2024-09-18T01:07:45.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6801999999999999e-05 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298674+0000", 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3300000000000005e-06 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298743+0000", 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9134000000000006e-05 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298768+0000", 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4691000000000001e-05 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303012+0000", 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042442219999999998 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303060+0000", 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8279000000000003e-05 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.139 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.8968917479999998, 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.140 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:07:45.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:07:45.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3130 3.a 3:5042a8b3:::rbd_data.11769efc726f.00000000000000a9:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.207273+0000", 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.8744692399999998, 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033564459999999999, 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.146 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3130 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207273+0000", 2024-09-18T01:07:45.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207273+0000", 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207275+0000", 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207282+0000", 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8249999999999999e-06 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207298+0000", 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5509e-05 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207305+0000", 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6699999999999994e-06 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207366+0000", 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0476999999999997e-05 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207456+0000", 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0051999999999998e-05 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.207572+0000", 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116295 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.208745+0000", 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011732190000000001 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.210524+0000", 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001778609 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.210585+0000", 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1467e-05 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.210629+0000", 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3979e-05 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3236 3.a 3:556bb06e:::rbd_data.11769efc726f.00000000000000cf:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7821961979999994, 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042631809999999996, 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3236 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299548+0000", 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299556+0000", 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0620000000000001e-06 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299573+0000", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7538999999999999e-05 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299585+0000", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1576999999999999e-05 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299654+0000", 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9573000000000006e-05 2024-09-18T01:07:45.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299747+0000", 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2046999999999995e-05 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299864+0000", 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011751 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300513+0000", 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064886399999999997 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303745+0000", 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032316319999999999 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303774+0000", 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9618e-05 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303809+0000", 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5027999999999997e-05 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3233 3.1e e29/23 3:7edc2e33:::rbd_data.11769efc726f.00000000000000cc:head v 29'12, mlcod=29'10)", 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.299983+0000", 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7817588670000006, 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035590740000000002, 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299983+0000", 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299983+0000", 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299984+0000", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8000000000000004e-07 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299991+0000", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4710000000000004e-06 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300007+0000", 2024-09-18T01:07:45.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5909e-05 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300013+0000", 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3409999999999999e-06 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300080+0000", 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6643999999999994e-05 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300101+0000", 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1145000000000002e-05 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303445+0000", 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003344053 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303542+0000", 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7631000000000001e-05 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3319 3.a 3:5581bfb7:::rbd_data.11769efc726f.0000000000000093:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.393818+0000", 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.6879246059999993, 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038247189999999999, 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3319 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393818+0000", 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393818+0000", 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393819+0000", 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1200000000000001e-07 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393823+0000", 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4190000000000004e-06 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393835+0000", 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2266000000000001e-05 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393843+0000", 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4460000000000004e-06 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393907+0000", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4156e-05 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.393993+0000", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6298999999999995e-05 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.394107+0000", 2024-09-18T01:07:45.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001142 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.394823+0000", 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071601100000000001 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.397578+0000", 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002754785 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.397614+0000", 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5667000000000003e-05 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.397642+0000", 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8558e-05 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:6650 3.10 3:0ceed6d5:::rbd_object_map.1176ec0d676b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:07:45.170 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:39.033228+0000", 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0485143600000004, 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036332740000000001, 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 6650 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033228+0000", 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033228+0000", 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033231+0000", 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6690000000000002e-06 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033239+0000", 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1589999999999993e-06 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033256+0000", 2024-09-18T01:07:45.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6963999999999999e-05 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033264+0000", 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9200000000000004e-06 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033325+0000", 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1439000000000006e-05 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.033331+0000", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6060000000000002e-06 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.034860+0000", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001529664 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.034953+0000", 2024-09-18T01:07:45.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2510999999999996e-05 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.035062+0000", 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109344 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.036324+0000", 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012621589999999999 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.036796+0000", 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047176999999999999 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.036833+0000", 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7298000000000001e-05 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:39.036861+0000", 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7770999999999999e-05 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8804 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:7 [sparse-read 1835008~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:45.178 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.632841+0000", 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4489008080000003, 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033866690000000001, 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8804 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632841+0000", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632841+0000", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632844+0000", 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.272e-06 2024-09-18T01:07:45.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632852+0000", 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1210000000000007e-06 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632872+0000", 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0316e-05 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632883+0000", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.128e-05 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.632971+0000", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7489000000000002e-05 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.633071+0000", 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100386 2024-09-18T01:07:45.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.636228+0000", 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031568049999999999 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8950 3.10 3:0a09e975:::rbd_data.1176ec0d676b.00000000000000e8:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.634764+0000", 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4469784570000002, 2024-09-18T01:07:45.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031999630000000001, 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8950 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634764+0000", 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634764+0000", 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634764+0000", 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634766+0000", 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1909999999999999e-06 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634774+0000", 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4529999999999996e-06 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634779+0000", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3739999999999997e-06 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.637867+0000", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003087832 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.637964+0000", 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6670999999999997e-05 2024-09-18T01:07:45.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8978 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4469527620000004, 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044104569999999996, 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8978 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634790+0000", 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634792+0000", 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4159999999999998e-06 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634800+0000", 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7999999999999999e-06 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634806+0000", 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.637992+0000", 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031862969999999998 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.638073+0000", 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1143000000000006e-05 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639200+0000", 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011267199999999999 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4202151949999999, 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:07:45.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9971 3.a 3:54e5bf34:::rbd_data.117611b60058.0000000000000006:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5064427199999999, 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053341009999999999, 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9971 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.200 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575302+0000", 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575757+0000", 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000454411 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575778+0000", 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1458999999999999e-05 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575789+0000", 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0927000000000001e-05 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575885+0000", 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5517e-05 2024-09-18T01:07:45.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575892+0000", 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3869999999999997e-06 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.577992+0000", 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020999360000000002 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.578124+0000", 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013169000000000001 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.578267+0000", 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014353599999999999 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.579585+0000", 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013182230000000001 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580526+0000", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00094041800000000005 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580585+0000", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9206000000000002e-05 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580634+0000", 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8477999999999999e-05 2024-09-18T01:07:45.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:45.207 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.33628295699999999, 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:07:45.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:07:45.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.334273824, 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:07:45.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:45.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:07:45.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:07:45.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:07:45.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:45.219 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:45.220 INFO:teuthology.orchestra.run.smithi120.stdout:468 trunc from 0xda786ab to 0x118e0d3 2024-09-18T01:07:45.220 INFO:teuthology.orchestra.run.smithi120.stdout:469 write 0x71a5670 thru 0x71af4c7 (0x9e58 bytes) 2024-09-18T01:07:45.221 INFO:teuthology.orchestra.run.smithi120.stdout:470 writesame 0x69c5e2d thru 0x69d4cac data_size 0x40(0xee80 bytes) 2024-09-18T01:07:45.221 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:45.218+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:45.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:45.222 INFO:teuthology.orchestra.run.smithi120.stdout:471 read 0x2db4a7e thru 0x2db9751 (0x4cd4 bytes) 2024-09-18T01:07:45.222 INFO:teuthology.orchestra.run.smithi120.stdout:472 read 0x170523f thru 0x170cc94 (0x7a56 bytes) 2024-09-18T01:07:45.222 INFO:teuthology.orchestra.run.smithi120.stdout:473 clone 5 order 22 su 65536 sc 13 2024-09-18T01:07:45.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:45.321 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:45.319+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:45.365 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:45.366 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:45.366 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:45.366 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:45.374 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:45.413 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:45.414 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:45.421 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:45.420+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:45.430 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:45.430 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:45.446 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:45.446 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:45.462 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:45.478 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:45.479 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:45.495 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:45.495 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:45.511 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:45.511 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:45.519 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:45.519 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:45.522 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:45.521+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:45.528 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:45.529 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:45.529 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:45.563 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:07:45.623 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:45.622+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:45.673 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:45.673 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:45.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:45.726 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:45.724+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:45.786 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:45.786 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:45.796 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:45.825 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:45.825+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:45.927 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:45.926+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:45.938 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:45.938 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:45.947 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:46.028 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:46.027+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:46.049 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.049 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:46.129 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:46.128+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:46.162 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.162 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.171 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:46.230 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:46.228+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:46.274 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.274 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.282 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:46.366 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:46.364+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:46.424 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.424 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.433 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:46.433 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:46.432+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:46.534 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:46.532+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:46.579 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.579 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:46.634 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:46.632+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:46.736 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:46.735+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:46.740 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.741 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.749 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:46.836 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:46.835+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:46.851 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.851 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.860 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:46.937 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:46.936+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:46.963 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:46.963 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:46.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:47.038 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:47.038+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:47.075 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.075 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:47.140 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:47.139+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:47.189 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.189 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.198 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:47.241 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:47.240+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:47.312 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.312 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:47.342 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:47.340+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.425 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:47.443 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:47.442+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:47.496 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:07:47.496 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:07:47.536 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.536 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.539 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:47.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:47.578 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:07:47.585 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:47.582+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:47.645 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:47.643+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:47.653 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:47.653 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:47.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:47.675 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-18T01:07:47.675 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:47.675 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:47.693 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:47.693 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:47.712 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:47.712 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:47.731 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:47.732 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:47.746 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:47.744+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:47.750 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:47.751 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:47.768 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:47.768 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:47.770 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:47.770 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:47.779 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:47.786 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:47.786 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:47.803 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:47.803 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:47.847 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:47.846+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.885 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860268741000001, 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:07:47.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.86022659, 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:07:47.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860193162, 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.895 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:07:47.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860155662, 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:07:47.899 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:07:47.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:07:47.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860134085, 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:07:47.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860114944999999, 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:07:47.908 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:07:47.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:07:47.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860094554, 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:07:47.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860065154000001, 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860036364000001, 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:07:47.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:07:47.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.860015668999999, 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:47.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.859996185, 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:07:47.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.859962274000001, 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:07:47.935 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:07:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:07:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.859926274999999, 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:07:47.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.775478303, 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:07:47.944 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:07:47.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.775468094000001, 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:07:47.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.2750668679999997, 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:07:47.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:07:47.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.2595806989999998, 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.959 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:07:47.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.251689979, 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:07:47.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:07:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.2256209760000001, 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:07:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:07:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.220452184, 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:07:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:47.979 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:47.980 INFO:teuthology.orchestra.run.smithi120.stdout:474 writesame 0x226e2e8 thru 0x227d5d6 data_size 0x1(0xf2ef bytes) 2024-09-18T01:07:47.980 INFO:teuthology.orchestra.run.smithi120.stdout:475 read 0xd1bb8f thru 0xd23e06 (0x8278 bytes) 2024-09-18T01:07:47.981 INFO:teuthology.orchestra.run.smithi120.stdout:476 read 0x269bef5 thru 0x269ce90 (0xf9c bytes) 2024-09-18T01:07:47.981 INFO:teuthology.orchestra.run.smithi120.stdout:477 write 0xa2d936 thru 0xa302d8 (0x29a3 bytes) 2024-09-18T01:07:47.981 INFO:teuthology.orchestra.run.smithi120.stdout:478 write 0x51e42f9 thru 0x51ee60c (0xa314 bytes) 2024-09-18T01:07:47.981 INFO:teuthology.orchestra.run.smithi120.stdout:479 clone 6 order 19 su 131072 sc 2 2024-09-18T01:07:47.981 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:47.981 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:47.979+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:48.081 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:48.080+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:48.124 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.124 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:48.124 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:48.124 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:48.182 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:48.181+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:48.235 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:48.235 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:48.282 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:48.280+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:48.384 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:48.382+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.304609006, 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:07:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:07:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:07:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1815 3.a 3:5642d443:::rbd_data.11769efc726f.000000000000002f:head [write 0~180224 [fadvise_sequential+fadvise_nocache],write 524288~245760 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.238469258, 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053689159999999996, 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1815 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147676+0000", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.147678+0000", 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-18T01:07:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148125+0000", 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000447157 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148138+0000", 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3664e-05 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148146+0000", 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.61e-06 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148218+0000", 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1854999999999996e-05 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148347+0000", 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012892599999999999 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.148516+0000", 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000168925 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152752+0000", 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042366449999999998 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152932+0000", 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017960200000000001 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.152993+0000", 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1162999999999997e-05 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.153045+0000", 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1625000000000001e-05 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2636 3.a 3:5581bfb7:::rbd_data.11769efc726f.0000000000000093:head [write 0~245760 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.145444789999999, 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043618010000000002, 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2636 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240700+0000", 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240702+0000", 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240918+0000", 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021542999999999999 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240934+0000", 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6192999999999999e-05 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.240941+0000", 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1940000000000001e-06 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241013+0000", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1555000000000002e-05 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241121+0000", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107772 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.241263+0000", 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141953 2024-09-18T01:07:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.242614+0000", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013513710000000001 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.244954+0000", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002340066 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245013+0000", 2024-09-18T01:07:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9358999999999999e-05 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.245062+0000", 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8869999999999998e-05 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:3022 3.1e e28/23 3:7bb42000:::rbd_data.11769efc726f.00000000000000e9:head v 28'8, mlcod=28'6)", 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.08761505, 2024-09-18T01:07:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045301040000000001, 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298530+0000", 2024-09-18T01:07:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298531+0000", 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.212e-06 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298648+0000", 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116434 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298665+0000", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6801999999999999e-05 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298674+0000", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3300000000000005e-06 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298743+0000", 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9134000000000006e-05 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.298768+0000", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4691000000000001e-05 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303012+0000", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042442219999999998 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303060+0000", 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8279000000000003e-05 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.201294709999999, 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:07:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3236 3.a 3:556bb06e:::rbd_data.11769efc726f.00000000000000cf:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.08659916, 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042631809999999996, 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3236 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299546+0000", 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299548+0000", 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299556+0000", 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0620000000000001e-06 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299573+0000", 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7538999999999999e-05 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299585+0000", 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1576999999999999e-05 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299654+0000", 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9573000000000006e-05 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299747+0000", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2046999999999995e-05 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.299864+0000", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011751 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.300513+0000", 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064886399999999997 2024-09-18T01:07:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303745+0000", 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032316319999999999 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303774+0000", 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9618e-05 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.303809+0000", 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5027999999999997e-05 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8978 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.7513557239999997, 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044104569999999996, 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8978 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634789+0000", 2024-09-18T01:07:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634790+0000", 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634792+0000", 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4159999999999998e-06 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634800+0000", 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7999999999999999e-06 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.634806+0000", 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.637992+0000", 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031862969999999998 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.638073+0000", 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1143000000000006e-05 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639200+0000", 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011267199999999999 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.7246181570000001, 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9971 3.a 3:54e5bf34:::rbd_data.117611b60058.0000000000000006:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:07:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.8108456820000001, 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053341009999999999, 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9971 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575299+0000", 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575302+0000", 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575757+0000", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000454411 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575778+0000", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1458999999999999e-05 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575789+0000", 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0927000000000001e-05 2024-09-18T01:07:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575885+0000", 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5517e-05 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.575892+0000", 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3869999999999997e-06 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.577992+0000", 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020999360000000002 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.578124+0000", 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013169000000000001 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.578267+0000", 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014353599999999999 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.579585+0000", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013182230000000001 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580526+0000", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00094041800000000005 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580585+0000", 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9206000000000002e-05 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:43.580634+0000", 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8477999999999999e-05 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.640685919, 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:07:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.638676786, 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:07:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:07:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13115 3.10 3:0c31a8f9:::rbd_data.117611b60058.000000000000008c:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.117488+0000", 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2686574159999999, 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049730570000000003, 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13115 2024-09-18T01:07:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117488+0000", 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117488+0000", 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117489+0000", 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.254e-06 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117494+0000", 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6e-06 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117508+0000", 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4165000000000001e-05 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117515+0000", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4760000000000001e-06 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117576+0000", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0993999999999999e-05 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.117668+0000", 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1583999999999996e-05 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.122461+0000", 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047929840000000001 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13180 3.10 3:08fa31b6:::rbd_data.117611b60058.00000000000000e1:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.145990+0000", 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2401556199999999, 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047397680000000001, 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13180 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.145990+0000", 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.145990+0000", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.145991+0000", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-07 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.145995+0000", 2024-09-18T01:07:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5460000000000002e-06 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.146005+0000", 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.146011+0000", 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3580000000000002e-06 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.146063+0000", 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2420999999999998e-05 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.146190+0000", 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012720299999999999 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.150729+0000", 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045388800000000003 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13185 3.a 3:51a12e8c:::rbd_data.117611b60058.00000000000000ee:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2379732360000002, 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00605274, 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13185 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148174+0000", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148179+0000", 2024-09-18T01:07:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148182+0000", 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2540000000000001e-06 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148734+0000", 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055139499999999999 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148870+0000", 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013613699999999999 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.154225+0000", 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053546829999999998 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13204 3.10 3:08526667:::rbd_data.117611b60058.000000000000010d:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.155996+0000", 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2301491969999998, 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052955609999999998, 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13204 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.155996+0000", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.155996+0000", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.155997+0000", 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7199999999999997e-07 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156001+0000", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8369999999999999e-06 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156014+0000", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3811999999999999e-05 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156022+0000", 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5499999999999997e-06 2024-09-18T01:07:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156076+0000", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3647999999999999e-05 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156197+0000", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173299999999999 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.161292+0000", 2024-09-18T01:07:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050941090000000003 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13205 3.10 3:0e7e7b8f:::rbd_data.117611b60058.000000000000010e:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.156047+0000", 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2300984879999999, 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053549000000000001, 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13205 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156047+0000", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156047+0000", 2024-09-18T01:07:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156047+0000", 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8299999999999996e-07 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156050+0000", 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1080000000000001e-06 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156059+0000", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8370000000000003e-06 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156066+0000", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5889999999999999e-06 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156527+0000", 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000460705 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.156686+0000", 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015974300000000001 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.161402+0000", 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047152349999999999 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13212 3.10 3:0e064f8d:::rbd_data.117611b60058.0000000000000119:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.158556+0000", 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2275895559999999, 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054713169999999998, 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13212 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158556+0000", 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158556+0000", 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158557+0000", 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.012e-06 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158562+0000", 2024-09-18T01:07:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0610000000000003e-06 2024-09-18T01:07:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158575+0000", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3057000000000001e-05 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158580+0000", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0869999999999997e-06 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158617+0000", 2024-09-18T01:07:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110999999999998e-05 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158667+0000", 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9947999999999997e-05 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.158736+0000", 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9513999999999999e-05 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.159212+0000", 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047542100000000001 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.163963+0000", 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047507870000000002 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.163995+0000", 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2511000000000001e-05 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.164027+0000", 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1807999999999998e-05 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13241 3.10 3:090d2a4d:::rbd_data.117611b60058.0000000000000149:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2142505649999999, 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005514878, 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13241 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171896+0000", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171908+0000", 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2401e-05 2024-09-18T01:07:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171918+0000", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.042e-05 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171925+0000", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171967+0000", 2024-09-18T01:07:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1257000000000002e-05 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.172085+0000", 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011883999999999999 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.177409+0000", 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053241210000000002 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13253 3.a 3:55279937:::rbd_data.117611b60058.0000000000000159:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.176539+0000", 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2096066360000002, 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053848050000000003, 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13253 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176539+0000", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176539+0000", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176539+0000", 2024-09-18T01:07:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8700000000000005e-07 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176543+0000", 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8569999999999997e-06 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176557+0000", 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3852000000000001e-05 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176565+0000", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3890000000000005e-06 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176609+0000", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3272999999999997e-05 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.176704+0000", 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5463000000000004e-05 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.181923+0000", 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005219184 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13269 3.a 3:5629e52b:::rbd_data.117611b60058.0000000000000175:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.182725+0000", 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.2034198649999999, 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047814149999999998, 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13269 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182725+0000", 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182725+0000", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182726+0000", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7899999999999997e-07 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182730+0000", 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8940000000000003e-06 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182740+0000", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.91e-06 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182745+0000", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1410000000000004e-06 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182789+0000", 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3445000000000002e-05 2024-09-18T01:07:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.182871+0000", 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2106000000000003e-05 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.187507+0000", 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0046360400000000001 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.497 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.498 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:48.498 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:48.497+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:48.577 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:07:48.577 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:07:48.598 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:07:48.605 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.605 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:48.605 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:48.605 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:48.639 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:48.637+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:48.700 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:48.699+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:48.717 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:48.717 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.725 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:48.802 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:48.800+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:07:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.971456013999997, 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:07:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:07:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:07:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.362166109, 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010713336, 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 140 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507131+0000", 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2549000000000003e-05 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507136+0000", 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5499999999999996e-06 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507139+0000", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512990+0000", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058514930000000001 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513036+0000", 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5995000000000003e-05 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513109+0000", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2559999999999996e-05 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514456+0000", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001347144 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517765+0000", 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00330846 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517791+0000", 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6574999999999999e-05 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517812+0000", 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0775999999999999e-05 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.362120066000003, 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010897142, 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 141 2024-09-18T01:07:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507148+0000", 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507179+0000", 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1504000000000002e-05 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507185+0000", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7089999999999999e-06 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513570+0000", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0063849830000000003 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513613+0000", 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3689000000000001e-05 2024-09-18T01:07:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513682+0000", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8750000000000004e-05 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514460+0000", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077749700000000004 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517975+0000", 2024-09-18T01:07:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035148150000000001 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518005+0000", 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9986000000000001e-05 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518042+0000", 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2792 3.15 3:a9574683:::rbd_data.1176f5693ee0.000000000000002c:3 [read 2392064~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.254562+0000", 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.614702855999999, 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098079770000000007, 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2792 2024-09-18T01:07:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254562+0000", 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254562+0000", 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254562+0000", 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254564+0000", 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254570+0000", 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.913e-06 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254574+0000", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.442e-06 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264250+0000", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096756719999999997 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264293+0000", 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2753000000000002e-05 2024-09-18T01:07:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264370+0000", 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7062000000000002e-05 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2793 3.15 3:a9574683:::rbd_data.1176f5693ee0.000000000000002c:3 [read 2523136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.254580+0000", 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.61468498, 2024-09-18T01:07:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099249899999999999, 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2793 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254580+0000", 2024-09-18T01:07:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254580+0000", 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254580+0000", 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254583+0000", 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254591+0000", 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9950000000000005e-06 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254595+0000", 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6140000000000002e-06 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264400+0000", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098051010000000001 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264447+0000", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6356999999999997e-05 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264505+0000", 2024-09-18T01:07:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8143999999999998e-05 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2794 3.15 3:a9574683:::rbd_data.1176f5693ee0.000000000000002c:3 [read 2654208~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.254601+0000", 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.614663696000001, 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010055602, 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2794 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254601+0000", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254601+0000", 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254601+0000", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254603+0000", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.694e-06 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254609+0000", 2024-09-18T01:07:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0530000000000003e-06 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254614+0000", 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.561e-06 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264536+0000", 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099222240000000003 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264588+0000", 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1722999999999999e-05 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264657+0000", 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8997000000000004e-05 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2795 3.15 3:a9574683:::rbd_data.1176f5693ee0.000000000000002c:3 [read 2785280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.254619+0000", 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.614645832000001, 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010260613999999999, 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2795 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254619+0000", 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254619+0000", 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254619+0000", 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254621+0000", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.761e-06 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254627+0000", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7270000000000004e-06 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254631+0000", 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6430000000000003e-06 2024-09-18T01:07:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264703+0000", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010071296 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264779+0000", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5951999999999994e-05 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264879+0000", 2024-09-18T01:07:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100833 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2796 3.15 3:a9574683:::rbd_data.1176f5693ee0.000000000000002c:3 [read 2916352~10350 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.254637+0000", 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.614628032000001, 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010564914999999999, 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2796 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254637+0000", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254637+0000", 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254637+0000", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254639+0000", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.81e-06 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254644+0000", 2024-09-18T01:07:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5729999999999998e-06 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.254650+0000", 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264917+0000", 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010267312000000001 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.264990+0000", 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3078999999999996e-05 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.265202+0000", 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000211392 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2946 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:snapdir [list-snaps] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.279754+0000", 2024-09-18T01:07:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.589511076000001, 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0097714059999999998, 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2946 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279754+0000", 2024-09-18T01:07:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279754+0000", 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279754+0000", 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279761+0000", 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.08e-06 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279768+0000", 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1760000000000004e-06 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.279773+0000", 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289424+0000", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096512319999999992 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289476+0000", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2049000000000003e-05 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289525+0000", 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8736999999999998e-05 2024-09-18T01:07:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:2952 3.d 3:b383eec4:::rbd_data.11769efc726f.00000000000000b5:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.283837+0000", 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.585427585, 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010222447000000001, 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2952 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.283837+0000", 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.283837+0000", 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.283838+0000", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000003e-07 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.284231+0000", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039348299999999997 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.284245+0000", 2024-09-18T01:07:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3719e-05 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.284255+0000", 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7559999999999994e-06 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289550+0000", 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052947610000000003 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289638+0000", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8121999999999994e-05 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.289770+0000", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013198600000000001 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.293605+0000", 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038354769999999999 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.294015+0000", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040954199999999999 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.294039+0000", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4329999999999999e-05 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.294060+0000", 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0394000000000002e-05 2024-09-18T01:07:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3013 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:3 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290877+0000", 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.578387941000001, 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0097109710000000005, 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3013 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290877+0000", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290877+0000", 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290877+0000", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1699999999999999e-07 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290879+0000", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.945e-06 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290885+0000", 2024-09-18T01:07:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9170000000000003e-06 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290892+0000", 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5089999999999999e-06 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300452+0000", 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095599710000000004 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300526+0000", 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4733000000000004e-05 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300588+0000", 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1478999999999999e-05 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3014 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:3 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290910+0000", 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.578355024, 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098305619999999993, 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3014 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290910+0000", 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290910+0000", 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290910+0000", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.89e-07 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290913+0000", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290922+0000", 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6889999999999999e-06 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290928+0000", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300620+0000", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096919689999999999 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300668+0000", 2024-09-18T01:07:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7944e-05 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300740+0000", 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2467999999999998e-05 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3015 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.578329541, 2024-09-18T01:07:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012252512, 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3015 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290936+0000", 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290938+0000", 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290949+0000", 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1538e-05 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290954+0000", 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8919999999999999e-06 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300757+0000", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098030359999999993 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300798+0000", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0899999999999998e-05 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303188+0000", 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002389689 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.578276716, 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:07:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:07:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:07:48.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8294 3.7 e34/23 3:e3e34f1e:::rbd_data.11762d3aac1.0000000000000000:head v 34'131, mlcod=31'129)", 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.2806318799999996, 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010999711000000001, 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:48.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588634+0000", 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.71e-07 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592677+0000", 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040433090000000001 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592690+0000", 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2716e-05 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592697+0000", 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9439999999999999e-06 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592766+0000", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9370999999999997e-05 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592798+0000", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1809e-05 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599587+0000", 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067890449999999996 2024-09-18T01:07:48.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599633+0000", 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646000000000003e-05 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.2115454159999999, 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:07:48.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:07:48.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12807 3.d 3:b5154276:::rbd_data.1176ec0d676b.0000000000000016:head [sparse-read 0~1252609] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.1224967220000002, 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011900506999999999, 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12807 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:48.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746769+0000", 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2100000000000001e-06 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746779+0000", 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0023e-05 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746789+0000", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0105000000000001e-05 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746803+0000", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3784999999999999e-05 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753526+0000", 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067230479999999997 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753625+0000", 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8485000000000003e-05 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758668+0000", 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050438510000000002 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12810 3.15 3:ae5c894f:::rbd_data.1176ec0d676b.0000000000000019:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.1224306799999999, 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01205553, 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12810 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1300000000000001e-07 2024-09-18T01:07:48.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746837+0000", 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3269999999999999e-06 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746846+0000", 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6690000000000001e-06 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746850+0000", 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8310000000000003e-06 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753576+0000", 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067258439999999999 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753682+0000", 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105919 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758890+0000", 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005207527 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12826 3.d 3:b63623b7:::rbd_data.1176ec0d676b.0000000000000029:head [sparse-read 0~1044659] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.1223935459999996, 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014765014999999999, 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12826 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746872+0000", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746874+0000", 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746884+0000", 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0088999999999999e-05 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746889+0000", 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.259e-06 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758702+0000", 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011813259 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758789+0000", 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6386999999999998e-05 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761636+0000", 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002847488 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.93215826700000004, 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:48.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:07:48.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:48.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:48.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:48.975 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:48.976 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:48.975+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:48.977 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:49.077 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:49.076+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:49.080 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:49.080 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:49.088 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:49.179 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:49.177+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:49.191 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:49.192 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:49.201 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:49.285 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:49.283+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:49.345 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.345 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:49.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1065 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274213782, 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017032160000000001, 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.346 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1065 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068778+0000", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068779+0000", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068790+0000", 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0781000000000001e-05 2024-09-18T01:07:49.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068794+0000", 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085655+0000", 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016861269000000002 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085720+0000", 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5431000000000004e-05 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085809+0000", 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8976999999999999e-05 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1066 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:49.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274191103, 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017957565000000002, 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1066 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068802+0000", 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068808+0000", 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8470000000000001e-06 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068812+0000", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1910000000000002e-06 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085836+0000", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017023470999999998 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085904+0000", 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8657000000000004e-05 2024-09-18T01:07:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086757+0000", 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085298500000000001 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1067 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274172179000001, 2024-09-18T01:07:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017422337, 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1067 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068821+0000", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1849999999999998e-06 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068827+0000", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0889999999999996e-06 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068832+0000", 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4170000000000002e-06 2024-09-18T01:07:49.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086073+0000", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017241587999999999 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086142+0000", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8508000000000002e-05 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086241+0000", 2024-09-18T01:07:49.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9174999999999998e-05 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1068 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274152939, 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017590855999999998, 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.360 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1068 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068840+0000", 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.939e-06 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068846+0000", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8719999999999999e-06 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068851+0000", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086267+0000", 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017416338 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086343+0000", 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6082999999999996e-05 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086429+0000", 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5807e-05 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1069 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274134305, 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017961016999999999, 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1069 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:49.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068859+0000", 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068865+0000", 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3249999999999996e-06 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068871+0000", 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8749999999999997e-06 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086452+0000", 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017580423000000001 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086716+0000", 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026484300000000001 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086818+0000", 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010118199999999999 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1070 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274113925, 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018106271, 2024-09-18T01:07:49.369 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1070 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068884+0000", 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6499999999999999e-06 2024-09-18T01:07:49.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068890+0000", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3239999999999999e-06 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068895+0000", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9280000000000001e-06 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086839+0000", 2024-09-18T01:07:49.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017944281999999999 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086905+0000", 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6000000000000005e-05 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086983+0000", 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7736000000000002e-05 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1071 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274088442, 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018256174, 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1071 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068903+0000", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068905+0000", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3800000000000001e-06 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068911+0000", 2024-09-18T01:07:49.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8680000000000004e-06 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068916+0000", 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5900000000000001e-06 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087005+0000", 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018089085000000001 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087072+0000", 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7306999999999998e-05 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087159+0000", 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6496999999999995e-05 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1072 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:49.378 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274068853999999, 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018416670999999999, 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1072 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068924+0000", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068940+0000", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5594999999999999e-05 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068946+0000", 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:07:49.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087184+0000", 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018238134999999999 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087253+0000", 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9287000000000003e-05 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087339+0000", 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5439999999999995e-05 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1073 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.274028621999999, 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018558742999999999, 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.383 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1073 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:49.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068963+0000", 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-07 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068965+0000", 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068972+0000", 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9120000000000001e-06 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068977+0000", 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7679999999999999e-06 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087363+0000", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018385865000000001 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087437+0000", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3930999999999994e-05 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087521+0000", 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4030999999999996e-05 2024-09-18T01:07:49.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7564282099999993, 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.388 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:49.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:07:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:49.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.394 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7527944570000002, 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:07:49.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:07:49.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7527708890000007, 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:07:49.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:49.404 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7511954870000004, 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:07:49.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:07:49.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8364 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7468271099999999, 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017169328000000001, 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8364 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.411 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596175+0000", 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1029000000000001e-05 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596183+0000", 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1359999999999997e-06 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596189+0000", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5770000000000001e-06 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610864+0000", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.014675349000000001 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610967+0000", 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010255399999999999 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613333+0000", 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002366253 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7467623959999994, 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:07:49.415 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:07:49.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:07:49.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7454402190000007, 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:49.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7413864080000003, 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:07:49.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7156979789999998, 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:07:49.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:07:49.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:49.438 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.6851597740000006, 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:07:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.6846700820000002, 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:07:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:07:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.449 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:49.451 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:49.451 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:49.449+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:49.538 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:07:49.551 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:49.550+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:49.557 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:07:49.621 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.621 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:49.621 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:49.633 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:49.652 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:49.651+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:49.754 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:49.752+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:49.787 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.787 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:49.787 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:49.787 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:49.787 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:49.796 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:49.819 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:49.854 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:49.853+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.766039585, 2024-09-18T01:07:49.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:07:49.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:49.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:07:49.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.955 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.766020781, 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.956 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:49.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:07:49.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:07:49.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.766004213, 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:49.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:07:49.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.971 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:49.972 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765988027000001, 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:07:49.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:07:49.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765976458000001, 2024-09-18T01:07:49.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:07:49.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:07:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:07:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765967092, 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:07:49.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:07:49.995 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:49.996 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765954909, 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:49.997 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:49.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:07:50.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:07:50.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765939213999999, 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:07:50.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:07:50.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:07:50.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:50.010 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765922059999999, 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:07:50.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:07:50.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:07:50.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765899894, 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:07:50.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765883444, 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:07:50.022 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765873989999999, 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:07:50.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:50.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:07:50.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765839938999999, 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:50.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:07:50.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:07:50.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:50.044 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765820215, 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:07:50.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:07:50.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.051 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765801978000001, 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:07:50.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:07:50.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765781403, 2024-09-18T01:07:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.060 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:07:50.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.066 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765762175999999, 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:50.072 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765742823, 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:07:50.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.765724168, 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:07:50.077 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:07:50.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:07:50.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:50.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:50.083 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.756942219000001, 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:50.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:07:50.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:50.089 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:50.090 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:50.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:50.091 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:50.092 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:50.092 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:50.092 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:50.093 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:50.093 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:50.093 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:50.093 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:50.094 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:50.094 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:50.094 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:50.095 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:50.095 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:50.095 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:50.095 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:50.096 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:50.096 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:50.089+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:50.192 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:50.190+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:50.233 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.233 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:50.292 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:50.290+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:50.345 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.345 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.354 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:50.395 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:50.394+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:50.494 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:50.493+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:50.497 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.498 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.507 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:50.591 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:07:50.595 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:50.594+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:50.598 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:07:50.608 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:07:50.614 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.614 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.625 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:50.693 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-18T01:07:50.696 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:50.694+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:50.798 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:50.796+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:50.798 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.799 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.809 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:50.837 INFO:teuthology.orchestra.run.smithi120.stdout:480 write 0x68bedbd thru 0x68ccc44 (0xde88 bytes) 2024-09-18T01:07:50.847 INFO:teuthology.orchestra.run.smithi120.stdout:481 read 0xd12c98 thru 0xd1dfdf (0xb348 bytes) 2024-09-18T01:07:50.848 INFO:teuthology.orchestra.run.smithi120.stdout:482 write 0x1daafed thru 0x1dba5da (0xf5ee bytes) 2024-09-18T01:07:50.851 INFO:teuthology.orchestra.run.smithi120.stdout:483 read 0x45308bb thru 0x453d067 (0xc7ad bytes) 2024-09-18T01:07:50.851 INFO:teuthology.orchestra.run.smithi120.stdout:484 read 0x108a10e thru 0x1094202 (0xa0f5 bytes) 2024-09-18T01:07:50.867 INFO:teuthology.orchestra.run.smithi120.stdout:486 write 0xc37a637 thru 0xc389cb0 (0xf67a bytes) 2024-09-18T01:07:50.876 INFO:teuthology.orchestra.run.smithi120.stdout:487 write 0x3f81ad9 thru 0x3f911d8 (0xf700 bytes) 2024-09-18T01:07:50.881 INFO:teuthology.orchestra.run.smithi120.stdout:488 write 0x2c49e12 thru 0x2c5948e (0xf67d bytes) 2024-09-18T01:07:50.884 INFO:teuthology.orchestra.run.smithi120.stdout:489 read 0x75c8b6d thru 0x75d7976 (0xee0a bytes) 2024-09-18T01:07:50.884 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:50.884 INFO:teuthology.orchestra.run.smithi120.stdout:492 write 0x80a01ee thru 0x80a890c (0x871f bytes) 2024-09-18T01:07:50.888 INFO:teuthology.orchestra.run.smithi120.stdout:493 write 0x823651 thru 0x8258d0 (0x2280 bytes) 2024-09-18T01:07:50.891 INFO:teuthology.orchestra.run.smithi120.stdout:495 write 0x9a5f306 thru 0x9a628fa (0x35f5 bytes) 2024-09-18T01:07:50.895 INFO:teuthology.orchestra.run.smithi120.stdout:496 write 0xa9383ae thru 0xa93d6cd (0x5320 bytes) 2024-09-18T01:07:50.899 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:50.898+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:50.899 INFO:teuthology.orchestra.run.smithi120.stdout:497 read 0xbf736ac thru 0xbf7a4d6 (0x6e2b bytes) 2024-09-18T01:07:50.900 INFO:teuthology.orchestra.run.smithi120.stdout:502 read 0x2d43293 thru 0x2d50691 (0xd3ff bytes) 2024-09-18T01:07:50.900 INFO:teuthology.orchestra.run.smithi120.stdout:503 write 0x5b70acb thru 0x5b7f7e5 (0xed1b bytes) 2024-09-18T01:07:50.912 INFO:teuthology.orchestra.run.smithi120.stdout:504 write 0xe530ede thru 0xe53f2a1 (0xe3c4 bytes) 2024-09-18T01:07:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:505 write 0x9ef791e thru 0x9ef9464 (0x1b47 bytes) 2024-09-18T01:07:50.919 INFO:teuthology.orchestra.run.smithi120.stdout:507 write 0x5d10852 thru 0x5d149a8 (0x4157 bytes) 2024-09-18T01:07:50.931 INFO:teuthology.orchestra.run.smithi120.stdout:508 trunc from 0xe53f2a2 to 0x717cc2c 2024-09-18T01:07:50.945 INFO:teuthology.orchestra.run.smithi120.stdout:509 writesame 0x55f7013 thru 0x56069cf data_size 0x65(0xf9bd bytes) 2024-09-18T01:07:50.948 INFO:teuthology.orchestra.run.smithi120.stdout:510 writesame 0x31c2d5f thru 0x31d1f34 data_size 0xa(0xf1d6 bytes) 2024-09-18T01:07:50.954 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:50.954 INFO:teuthology.orchestra.run.smithi120.stdout:513 write 0x5935ce thru 0x599d98 (0x67cb bytes) 2024-09-18T01:07:50.958 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:50.958 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:50.971 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:50.972 INFO:teuthology.orchestra.run.smithi120.stdout:514 writesame 0xd9efb12 thru 0xd9f91bb data_size 0x26(0x96aa bytes) 2024-09-18T01:07:50.972 INFO:teuthology.orchestra.run.smithi120.stdout:515 write 0xd0f4073 thru 0xd10086c (0xc7fa bytes) 2024-09-18T01:07:50.975 INFO:teuthology.orchestra.run.smithi120.stdout:516 read 0x9644e88 thru 0x9654b15 (0xfc8e bytes) 2024-09-18T01:07:50.980 INFO:teuthology.orchestra.run.smithi120.stdout:517 writesame 0xdb9420e thru 0xdb95dd8 data_size 0x1(0x1bcb bytes) 2024-09-18T01:07:50.980 INFO:teuthology.orchestra.run.smithi120.stdout:518 punch from 0xcefd345 to 0xcf085b7, (0xb272 bytes) 2024-09-18T01:07:50.980 INFO:teuthology.orchestra.run.smithi120.stdout:521 read 0xd7dae7c thru 0xd7db1b4 (0x339 bytes) 2024-09-18T01:07:50.980 INFO:teuthology.orchestra.run.smithi120.stdout:524 writesame 0x4138d9a thru 0x41451da data_size 0x1(0xc441 bytes) 2024-09-18T01:07:50.995 INFO:teuthology.orchestra.run.smithi120.stdout:526 write 0x8daac12 thru 0x8db6c2f (0xc01e bytes) 2024-09-18T01:07:50.999 INFO:teuthology.orchestra.run.smithi120.stdout:527 read 0x5c73436 thru 0x5c7a5f6 (0x71c1 bytes) 2024-09-18T01:07:50.999 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:50.999 INFO:teuthology.orchestra.run.smithi120.stdout:529 read 0x1d68f18 thru 0x1d6f349 (0x6432 bytes) 2024-09-18T01:07:50.999 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:50.997+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:51.008 INFO:teuthology.orchestra.run.smithi120.stdout:531 trunc from 0xdb95dd9 to 0x5cbf9fc 2024-09-18T01:07:51.021 INFO:teuthology.orchestra.run.smithi120.stdout:532 read 0x18242d7 thru 0x1833159 (0xee83 bytes) 2024-09-18T01:07:51.021 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:51.031 INFO:teuthology.orchestra.run.smithi120.stdout:535 writesame 0x9e81a69 thru 0x9e8fc27 data_size 0x1(0xe1bf bytes) 2024-09-18T01:07:51.031 INFO:teuthology.orchestra.run.smithi120.stdout:536 read 0x98c313e thru 0x98c7874 (0x4737 bytes) 2024-09-18T01:07:51.031 INFO:teuthology.orchestra.run.smithi120.stdout:537 write 0x32a924b thru 0x32ae5ed (0x53a3 bytes) 2024-09-18T01:07:51.034 INFO:teuthology.orchestra.run.smithi120.stdout:542 read 0x6f886a2 thru 0x6f94f5e (0xc8bd bytes) 2024-09-18T01:07:51.037 INFO:teuthology.orchestra.run.smithi120.stdout:543 trunc from 0x9e8fc28 to 0x772e4b6 2024-09-18T01:07:51.042 INFO:teuthology.orchestra.run.smithi120.stdout:544 read 0x6c37c13 thru 0x6c3988b (0x1c79 bytes) 2024-09-18T01:07:51.042 INFO:teuthology.orchestra.run.smithi120.stdout:546 read 0x2ed0f8d thru 0x2ed4567 (0x35db bytes) 2024-09-18T01:07:51.049 INFO:teuthology.orchestra.run.smithi120.stdout:547 trunc from 0x772e4b6 to 0x155b0ba 2024-09-18T01:07:51.087 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.087 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.089 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:51.096 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:51.109 INFO:teuthology.orchestra.run.smithi120.stdout:549 writesame 0xd78cace thru 0xd79281e data_size 0x1(0x5d51 bytes) 2024-09-18T01:07:51.109 INFO:teuthology.orchestra.run.smithi120.stdout:550 punch from 0x64d0df6 to 0x64d6b33, (0x5d3d bytes) 2024-09-18T01:07:51.109 INFO:teuthology.orchestra.run.smithi120.stdout:551 punch from 0x73e5f54 to 0x73e6a9a, (0xb46 bytes) 2024-09-18T01:07:51.109 INFO:teuthology.orchestra.run.smithi120.stdout:552 read 0x415bb8d thru 0x4161af0 (0x5f64 bytes) 2024-09-18T01:07:51.111 INFO:teuthology.orchestra.run.smithi120.stdout:553 trunc from 0xd79281f to 0xbb82b0f 2024-09-18T01:07:51.116 INFO:teuthology.orchestra.run.smithi120.stdout:554 punch from 0x6800c85 to 0x6809bca, (0x8f45 bytes) 2024-09-18T01:07:51.116 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:51.116 INFO:teuthology.orchestra.run.smithi120.stdout:556 punch from 0x1d74b17 to 0x1d7896a, (0x3e53 bytes) 2024-09-18T01:07:51.116 INFO:teuthology.orchestra.run.smithi120.stdout:557 write 0x75b4acf thru 0x75b85eb (0x3b1d bytes) 2024-09-18T01:07:51.119 INFO:teuthology.orchestra.run.smithi120.stdout:558 read 0xd2921 thru 0xd6616 (0x3cf6 bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:559 read 0x9e2058c thru 0x9e28ac6 (0x853b bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:560 punch from 0xafdb559 to 0xafe04c7, (0x4f6e bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:561 read 0x549ebf7 thru 0x54ac755 (0xdb5f bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:562 punch from 0x4b6a0f1 to 0x4b6f2fd, (0x520c bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:563 read 0x517cb80 thru 0x518874e (0xbbcf bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:564 punch from 0x52c3ca2 to 0x52d20be, (0xe41c bytes) 2024-09-18T01:07:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:565 read 0xb121e92 thru 0xb122678 (0x7e7 bytes) 2024-09-18T01:07:51.121 INFO:teuthology.orchestra.run.smithi120.stdout:566 read 0x96a0a80 thru 0x96aef3a (0xe4bb bytes) 2024-09-18T01:07:51.121 INFO:teuthology.orchestra.run.smithi120.stdout:567 write 0x215e689 thru 0x2161642 (0x2fba bytes) 2024-09-18T01:07:51.125 INFO:teuthology.orchestra.run.smithi120.stdout:568 clone 7 order 19 su 32768 sc 7 2024-09-18T01:07:51.141 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:51.140+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:51.200 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.200 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.201 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:51.200+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:51.209 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:51.302 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:51.300+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:51.353 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.353 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.361 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:51.403 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:51.402+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:51.464 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.464 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.472 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:51.503 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:51.503+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:51.575 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.575 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:51.592 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:07:51.595 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:07:51.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:07:51.627 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:51.626+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:51.689 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.690 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:51.707 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:51.706+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:51.808 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:51.807+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:51.842 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.843 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:51.851 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:51.935 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:51.934+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:51.995 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:51.995 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:52.003 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:52.044 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:52.043+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:52.097 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:52.141 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:52.140+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:52.146 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:52.146 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:52.155 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:52.211 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:52.210+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:52.216 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:52.216 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:52.233 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:52.233 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:52.248 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:52.248 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:52.258 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:52.258 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:52.264 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:52.264 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:52.267 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:52.280 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:52.281 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:52.296 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:52.296 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:52.312 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:52.313 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:52.313 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:52.312+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:52.329 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:52.329 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:52.411 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:52.412 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:52.412 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:52.412 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:52.414 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:52.413+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:52.420 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:52.515 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:52.514+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.664912772999998, 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:07:52.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:07:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:07:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.055622868, 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010713336, 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 140 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507099+0000", 2024-09-18T01:07:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507131+0000", 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2549000000000003e-05 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507136+0000", 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5499999999999996e-06 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507139+0000", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.512990+0000", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058514930000000001 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513036+0000", 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5995000000000003e-05 2024-09-18T01:07:52.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513109+0000", 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2559999999999996e-05 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514456+0000", 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001347144 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517765+0000", 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00330846 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517791+0000", 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6574999999999999e-05 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517812+0000", 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0775999999999999e-05 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.055576825000003, 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010897142, 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:07:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 141 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507145+0000", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507148+0000", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507179+0000", 2024-09-18T01:07:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1504000000000002e-05 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.507185+0000", 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7089999999999999e-06 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513570+0000", 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0063849830000000003 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513613+0000", 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3689000000000001e-05 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.513682+0000", 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8750000000000004e-05 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.514460+0000", 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077749700000000004 2024-09-18T01:07:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.517975+0000", 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035148150000000001 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518005+0000", 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9986000000000001e-05 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:13.518042+0000", 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3015 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.271786299999999, 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012252512, 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3015 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290936+0000", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290938+0000", 2024-09-18T01:07:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290949+0000", 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1538e-05 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290954+0000", 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8919999999999999e-06 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300757+0000", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098030359999999993 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300798+0000", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0899999999999998e-05 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303188+0000", 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002389689 2024-09-18T01:07:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.271733475000001, 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:07:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:07:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8294 3.7 e34/23 3:e3e34f1e:::rbd_data.11762d3aac1.0000000000000000:head v 34'131, mlcod=31'129)", 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 11.974088639, 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010999711000000001, 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588633+0000", 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.588634+0000", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.71e-07 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592677+0000", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040433090000000001 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592690+0000", 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2716e-05 2024-09-18T01:07:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592697+0000", 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9439999999999999e-06 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592766+0000", 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9370999999999997e-05 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.592798+0000", 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1809e-05 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599587+0000", 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067890449999999996 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599633+0000", 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646000000000003e-05 2024-09-18T01:07:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 11.905002175, 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:07:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12807 3.d 3:b5154276:::rbd_data.1176ec0d676b.0000000000000016:head [sparse-read 0~1252609] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.8159534810000002, 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011900506999999999, 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12807 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746769+0000", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2100000000000001e-06 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746779+0000", 2024-09-18T01:07:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0023e-05 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746789+0000", 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0105000000000001e-05 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746803+0000", 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3784999999999999e-05 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753526+0000", 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067230479999999997 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753625+0000", 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8485000000000003e-05 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758668+0000", 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050438510000000002 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12810 3.15 3:ae5c894f:::rbd_data.1176ec0d676b.0000000000000019:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.8158874389999999, 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01205553, 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12810 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1300000000000001e-07 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746837+0000", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3269999999999999e-06 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746846+0000", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6690000000000001e-06 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746850+0000", 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8310000000000003e-06 2024-09-18T01:07:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753576+0000", 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067258439999999999 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753682+0000", 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105919 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758890+0000", 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005207527 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12826 3.d 3:b63623b7:::rbd_data.1176ec0d676b.0000000000000029:head [sparse-read 0~1044659] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.8158503049999997, 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014765014999999999, 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12826 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746872+0000", 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746874+0000", 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746884+0000", 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0088999999999999e-05 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746889+0000", 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.259e-06 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758702+0000", 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011813259 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758789+0000", 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6386999999999998e-05 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761636+0000", 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002847488 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6256150260000002, 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:07:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:07:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15117 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.0038050840000001, 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012024517, 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15117 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558917+0000", 2024-09-18T01:07:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499999999999997e-07 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558923+0000", 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558935+0000", 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2143e-05 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558941+0000", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7349999999999999e-06 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561319+0000", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023776959999999999 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561407+0000", 2024-09-18T01:07:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8103999999999995e-05 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561647+0000", 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239454 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570808+0000", 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0091611109999999996 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570822+0000", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3788000000000001e-05 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570886+0000", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3975999999999999e-05 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570941+0000", 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5313999999999999e-05 2024-09-18T01:07:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15118 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.559475+0000", 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.0032465589999999, 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011663957000000001, 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15118 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559475+0000", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559475+0000", 2024-09-18T01:07:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559476+0000", 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0200000000000001e-07 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559480+0000", 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559493+0000", 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2157999999999999e-05 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559498+0000", 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6230000000000002e-06 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.562150+0000", 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026514450000000001 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.562245+0000", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5787999999999998e-05 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.562523+0000", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027775899999999998 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570955+0000", 2024-09-18T01:07:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084321689999999998 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571039+0000", 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3462999999999997e-05 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571094+0000", 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5371999999999998e-05 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571139+0000", 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4647000000000001e-05 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15119 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.559644+0000", 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.0030779019999998, 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011629442, 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15119 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559644+0000", 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559644+0000", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559645+0000", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.052e-06 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559651+0000", 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:07:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559666+0000", 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5126000000000001e-05 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.559671+0000", 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.562849+0000", 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003177657 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.562948+0000", 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8580999999999997e-05 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.563396+0000", 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000448252 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570961+0000", 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0075649879999999999 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571194+0000", 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232847 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571227+0000", 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3027000000000001e-05 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571273+0000", 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6236000000000003e-05 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15126 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.566860+0000", 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9958614260000003, 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010698043000000001, 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15126 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.566860+0000", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.566860+0000", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.566861+0000", 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8800000000000002e-07 2024-09-18T01:07:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567443+0000", 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058208000000000001 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567453+0000", 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9089999999999995e-06 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567459+0000", 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8560000000000003e-06 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567543+0000", 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4237e-05 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567629+0000", 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5884000000000006e-05 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.567714+0000", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5167999999999995e-05 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577483+0000", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0097693220000000004 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577492+0000", 2024-09-18T01:07:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2390000000000002e-06 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577526+0000", 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4912e-05 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577558+0000", 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1647999999999999e-05 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15128 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9912258080000003, 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011752311999999999, 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15128 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9100000000000003e-07 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572019+0000", 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052237400000000004 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572034+0000", 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4963999999999999e-05 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572041+0000", 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9580000000000001e-06 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572094+0000", 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3115999999999998e-05 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572230+0000", 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013626 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572378+0000", 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014820799999999999 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577599+0000", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052203060000000001 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583147+0000", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055480829999999997 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583211+0000", 2024-09-18T01:07:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4766999999999998e-05 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583248+0000", 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6485000000000001e-05 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15386 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.599561+0000", 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9631600339999999, 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011302045, 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15386 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599561+0000", 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599561+0000", 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599562+0000", 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3900000000000003e-07 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599568+0000", 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599579+0000", 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0920999999999999e-05 2024-09-18T01:07:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.599590+0000", 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0986e-05 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.604158+0000", 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045681910000000001 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.604249+0000", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0941999999999998e-05 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.604489+0000", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024051899999999999 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610679+0000", 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061893399999999998 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610754+0000", 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5168999999999998e-05 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610821+0000", 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6945000000000004e-05 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610863+0000", 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2833000000000001e-05 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15388 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.600381+0000", 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9623399799999999, 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010638168, 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15388 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600381+0000", 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600381+0000", 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600382+0000", 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5000000000000002e-07 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600387+0000", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3460000000000004e-06 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600394+0000", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5000000000000002e-06 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.600399+0000", 2024-09-18T01:07:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.605404+0000", 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050052899999999999 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.605489+0000", 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5049000000000005e-05 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.605824+0000", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033449199999999998 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610686+0000", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048618430000000002 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610927+0000", 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000241287 2024-09-18T01:07:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.610979+0000", 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2126000000000002e-05 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.611020+0000", 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0680999999999999e-05 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:16053 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.647360+0000", 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9153615469999998, 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010911697999999999, 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 16053 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647360+0000", 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647360+0000", 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647361+0000", 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.92e-07 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647365+0000", 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2540000000000003e-06 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647373+0000", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2239999999999995e-06 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647378+0000", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.652242+0000", 2024-09-18T01:07:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048646490000000004 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.652305+0000", 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2623e-05 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.652627+0000", 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032164599999999998 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658174+0000", 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055471660000000001 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658217+0000", 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3167999999999997e-05 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658247+0000", 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9510999999999999e-05 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658272+0000", 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5058999999999998e-05 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:16054 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.647495+0000", 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.9152269039999998, 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010828231000000001, 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 16054 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647495+0000", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647495+0000", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647495+0000", 2024-09-18T01:07:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4600000000000003e-07 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647500+0000", 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647510+0000", 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8030000000000008e-06 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.647517+0000", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.472e-06 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.653019+0000", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055023609999999999 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.653077+0000", 2024-09-18T01:07:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7998999999999998e-05 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.653283+0000", 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000205553 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658210+0000", 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049270779999999997 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658278+0000", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8258999999999998e-05 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658303+0000", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5343000000000001e-05 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.658323+0000", 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9582e-05 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:52.691 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:52.693 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:07:52.693 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:52.691+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:52.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:07:52.694 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:52.793 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:52.792+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:52.800 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:52.801 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:52.811 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:52.893 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:52.893+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:52.954 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:52.954 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:52.955 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:52.955 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:52.955 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:52.963 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:52.995 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:52.994+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1065 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998112582000001, 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017032160000000001, 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1065 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068777+0000", 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068778+0000", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068779+0000", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068790+0000", 2024-09-18T01:07:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0781000000000001e-05 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068794+0000", 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085655+0000", 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016861269000000002 2024-09-18T01:07:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085720+0000", 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5431000000000004e-05 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085809+0000", 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8976999999999999e-05 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1066 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998089903, 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017957565000000002, 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1066 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068802+0000", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068808+0000", 2024-09-18T01:07:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8470000000000001e-06 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068812+0000", 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1910000000000002e-06 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085836+0000", 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017023470999999998 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085904+0000", 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8657000000000004e-05 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086757+0000", 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085298500000000001 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1067 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998070979000001, 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017422337, 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1067 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068821+0000", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1849999999999998e-06 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068827+0000", 2024-09-18T01:07:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0889999999999996e-06 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068832+0000", 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4170000000000002e-06 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086073+0000", 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017241587999999999 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086142+0000", 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8508000000000002e-05 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086241+0000", 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9174999999999998e-05 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1068 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998051739000001, 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017590855999999998, 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1068 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068840+0000", 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.939e-06 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068846+0000", 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8719999999999999e-06 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068851+0000", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086267+0000", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017416338 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086343+0000", 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6082999999999996e-05 2024-09-18T01:07:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086429+0000", 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5807e-05 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1069 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998033105000001, 2024-09-18T01:07:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017961016999999999, 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1069 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068859+0000", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068865+0000", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3249999999999996e-06 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068871+0000", 2024-09-18T01:07:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8749999999999997e-06 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086452+0000", 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017580423000000001 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086716+0000", 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026484300000000001 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086818+0000", 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010118199999999999 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1070 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.998012724999999, 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018106271, 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1070 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068884+0000", 2024-09-18T01:07:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6499999999999999e-06 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068890+0000", 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3239999999999999e-06 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068895+0000", 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9280000000000001e-06 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086839+0000", 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017944281999999999 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086905+0000", 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6000000000000005e-05 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086983+0000", 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7736000000000002e-05 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1071 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.997987242000001, 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018256174, 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1071 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068903+0000", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068905+0000", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3800000000000001e-06 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068911+0000", 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8680000000000004e-06 2024-09-18T01:07:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068916+0000", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5900000000000001e-06 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087005+0000", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018089085000000001 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087072+0000", 2024-09-18T01:07:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7306999999999998e-05 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087159+0000", 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6496999999999995e-05 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1072 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.997967654, 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018416670999999999, 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1072 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068924+0000", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068940+0000", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5594999999999999e-05 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068946+0000", 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:07:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087184+0000", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018238134999999999 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087253+0000", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9287000000000003e-05 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087339+0000", 2024-09-18T01:07:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5439999999999995e-05 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1073 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.997927422, 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018558742999999999, 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1073 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:07:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068963+0000", 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-07 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068965+0000", 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068972+0000", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9120000000000001e-06 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068977+0000", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7679999999999999e-06 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087363+0000", 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018385865000000001 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087437+0000", 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3930999999999994e-05 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087521+0000", 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4030999999999996e-05 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.48032701, 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:07:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:07:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.476693257000001, 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:07:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.476669689, 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:07:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.475094286999999, 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:07:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:07:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8364 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.470725910000001, 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017169328000000001, 2024-09-18T01:07:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8364 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596164+0000", 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596175+0000", 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1029000000000001e-05 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596183+0000", 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1359999999999997e-06 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596189+0000", 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5770000000000001e-06 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610864+0000", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.014675349000000001 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610967+0000", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010255399999999999 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613333+0000", 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002366253 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.470661196, 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:07:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.469339019, 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.145 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:07:53.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.465285207999999, 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:07:53.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.439596779, 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:07:53.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:07:53.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:07:53.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.409058573999999, 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:07:53.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:07:53.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.408568882000001, 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:07:53.168 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:07:53.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:07:53.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.173 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:53.174 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:53.175 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:53.173+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:53.276 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:53.274+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:53.318 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:53.319 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:53.319 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:53.319 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:53.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:53.376 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:53.376+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:53.470 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:53.470 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:53.470 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:53.470 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:53.470 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:53.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:53.478 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:53.476+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:53.578 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:53.577+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:53.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:07:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446496271000001, 2024-09-18T01:07:53.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:07:53.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:07:53.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:07:53.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446477467000001, 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:07:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:07:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446460899000002, 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.640 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:07:53.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446444712999998, 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:07:53.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:07:53.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446433144, 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.650 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:07:53.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:53.654 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446423778, 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:07:53.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:07:53.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446411595000001, 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446395899999999, 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:07:53.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:07:53.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.668 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446378746000001, 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.669 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:07:53.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.673 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.44635658, 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:07:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:07:53.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.678 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446340129999999, 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:07:53.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.683 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446330675999999, 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:07:53.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.688 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446296624999999, 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.689 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:07:53.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446276901000001, 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.694 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:07:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.698 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446258663999998, 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.699 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:07:53.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:53.703 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446238089000001, 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:07:53.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:07:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:53.708 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446218861999998, 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:07:53.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446199508999999, 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:07:53.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.717 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.446180854000001, 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:07:53.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:07:53.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:53.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.437398904999998, 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:53.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:07:53.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:53.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:53.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:53.731 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:53.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:07:53.734 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:53.734 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:53.731+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:53.765 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-18T01:07:53.834 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:53.833+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:53.889 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:53.889 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:53.889 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:53.890 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:53.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:53.903 INFO:teuthology.orchestra.run.smithi120.stdout:569 write 0x5a70961 thru 0x5a729ae (0x204e bytes) 2024-09-18T01:07:53.914 INFO:teuthology.orchestra.run.smithi120.stdout:570 write 0x6fe0e4e thru 0x6fe243f (0x15f2 bytes) 2024-09-18T01:07:53.921 INFO:teuthology.orchestra.run.smithi120.stdout:571 trunc from 0xbb82b0f to 0x955d9bc 2024-09-18T01:07:53.927 INFO:teuthology.orchestra.run.smithi120.stdout:573 writesame 0x1f18e43 thru 0x1f1f51a data_size 0xc(0x66d8 bytes) 2024-09-18T01:07:53.931 INFO:teuthology.orchestra.run.smithi120.stdout:576 write 0x7dfb21d thru 0x7e090cd (0xdeb1 bytes) 2024-09-18T01:07:53.934 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:53.933+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:580 read 0x334dbcb thru 0x335395e (0x5d94 bytes) 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:583 read 0x3c9a023 thru 0x3ca1c26 (0x7c04 bytes) 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:587 read 0x55106b6 thru 0x551116c (0xab7 bytes) 2024-09-18T01:07:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:589 read 0x4c7e3f2 thru 0x4c8d019 (0xec28 bytes) 2024-09-18T01:07:53.951 INFO:teuthology.orchestra.run.smithi120.stdout:590 writesame 0xe14daba thru 0xe15314e data_size 0x1(0x5695 bytes) 2024-09-18T01:07:53.951 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:53.951 INFO:teuthology.orchestra.run.smithi120.stdout:592 write 0xa9b88a1 thru 0xa9c05ed (0x7d4d bytes) 2024-09-18T01:07:53.958 INFO:teuthology.orchestra.run.smithi120.stdout:594 read 0x3d2653a thru 0x3d34e2c (0xe8f3 bytes) 2024-09-18T01:07:53.964 INFO:teuthology.orchestra.run.smithi120.stdout:595 write 0xe5fd458 thru 0xe605e41 (0x89ea bytes) 2024-09-18T01:07:53.969 INFO:teuthology.orchestra.run.smithi120.stdout:596 write 0x3807a41 thru 0x38120e1 (0xa6a1 bytes) 2024-09-18T01:07:53.977 INFO:teuthology.orchestra.run.smithi120.stdout:597 write 0xdd2f2bf thru 0xdd37ff6 (0x8d38 bytes) 2024-09-18T01:07:53.993 INFO:teuthology.orchestra.run.smithi120.stdout:598 trunc from 0xe605e42 to 0xbdbba16 2024-09-18T01:07:54.003 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.003 INFO:teuthology.orchestra.run.smithi120.stdout:600 read 0x167e889 thru 0x1686c43 (0x83bb bytes) 2024-09-18T01:07:54.003 INFO:teuthology.orchestra.run.smithi120.stdout:601 write 0xbc0a98 thru 0xbc6ab5 (0x601e bytes) 2024-09-18T01:07:54.006 INFO:teuthology.orchestra.run.smithi120.stdout:602 read 0x89cf714 thru 0x89de110 (0xe9fd bytes) 2024-09-18T01:07:54.006 INFO:teuthology.orchestra.run.smithi120.stdout:603 read 0x44ff6ae thru 0x4509580 (0x9ed3 bytes) 2024-09-18T01:07:54.006 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.006 INFO:teuthology.orchestra.run.smithi120.stdout:605 write 0x1caa7b3 thru 0x1cb4982 (0xa1d0 bytes) 2024-09-18T01:07:54.013 INFO:teuthology.orchestra.run.smithi120.stdout:606 writesame 0x614ff34 thru 0x615d3bc data_size 0x1(0xd489 bytes) 2024-09-18T01:07:54.020 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:54.020 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:54.020 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:54.020 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:54.021 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:54.030 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:54.034 INFO:teuthology.orchestra.run.smithi120.stdout:607 write 0x9c66f71 thru 0x9c7082a (0x98ba bytes) 2024-09-18T01:07:54.035 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:54.034+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:54.039 INFO:teuthology.orchestra.run.smithi120.stdout:608 punch from 0x96ab4a2 to 0x96ad01a, (0x1b78 bytes) 2024-09-18T01:07:54.039 INFO:teuthology.orchestra.run.smithi120.stdout:609 writesame 0x1cadace thru 0x1cb89f4 data_size 0x1(0xaf27 bytes) 2024-09-18T01:07:54.056 INFO:teuthology.orchestra.run.smithi120.stdout:610 read 0x5d85ec2 thru 0x5d880c4 (0x2203 bytes) 2024-09-18T01:07:54.056 INFO:teuthology.orchestra.run.smithi120.stdout:611 writesame 0xa83a40f thru 0xa83d0c3 data_size 0xf(0x2cb5 bytes) 2024-09-18T01:07:54.059 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.059 INFO:teuthology.orchestra.run.smithi120.stdout:613 read 0x304d978 thru 0x3057119 (0x97a2 bytes) 2024-09-18T01:07:54.059 INFO:teuthology.orchestra.run.smithi120.stdout:615 writesame 0xaf54fbe thru 0xaf59d3f data_size 0x3(0x4d82 bytes) 2024-09-18T01:07:54.065 INFO:teuthology.orchestra.run.smithi120.stdout:616 write 0x3d9fa09 thru 0x3da5bb1 (0x61a9 bytes) 2024-09-18T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:618 punch from 0x4deb63 to 0x4e4016, (0x54b3 bytes) 2024-09-18T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:619 read 0x68f078c thru 0x68fd9c9 (0xd23e bytes) 2024-09-18T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:620 read 0xa8c8eb7 thru 0xa8d7f9c (0xf0e6 bytes) 2024-09-18T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:621 punch from 0x3f12df9 to 0x3f1b656, (0x885d bytes) 2024-09-18T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:622 write 0x7e08716 thru 0x7e14807 (0xc0f2 bytes) 2024-09-18T01:07:54.073 INFO:teuthology.orchestra.run.smithi120.stdout:623 writesame 0x2719262 thru 0x27231bc data_size 0x29(0x9f5b bytes) 2024-09-18T01:07:54.077 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.077 INFO:teuthology.orchestra.run.smithi120.stdout:625 read 0x980447a thru 0x98076d3 (0x325a bytes) 2024-09-18T01:07:54.077 INFO:teuthology.orchestra.run.smithi120.stdout:626 write 0xbd828ca thru 0xbd84c90 (0x23c7 bytes) 2024-09-18T01:07:54.080 INFO:teuthology.orchestra.run.smithi120.stdout:627 punch from 0x1af3225 to 0x1af80cf, (0x4eaa bytes) 2024-09-18T01:07:54.080 INFO:teuthology.orchestra.run.smithi120.stdout:628 read 0x86c6499 thru 0x86d1af5 (0xb65d bytes) 2024-09-18T01:07:54.080 INFO:teuthology.orchestra.run.smithi120.stdout:629 punch from 0x8008e02 to 0x800b4e9, (0x26e7 bytes) 2024-09-18T01:07:54.080 INFO:teuthology.orchestra.run.smithi120.stdout:630 writesame 0x4a6d818 thru 0x4a726c4 data_size 0x1(0x4ead bytes) 2024-09-18T01:07:54.086 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.086 INFO:teuthology.orchestra.run.smithi120.stdout:633 writesame 0x74f1870 thru 0x74f8d77 data_size 0x38(0x7508 bytes) 2024-09-18T01:07:54.088 INFO:teuthology.orchestra.run.smithi120.stdout:634 punch from 0x3c3bf42 to 0x3c46e6c, (0xaf2a bytes) 2024-09-18T01:07:54.089 INFO:teuthology.orchestra.run.smithi120.stdout:635 write 0x6949337 thru 0x695397d (0xa647 bytes) 2024-09-18T01:07:54.093 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xee4bb0d 2024-09-18T01:07:54.096 INFO:teuthology.orchestra.run.smithi120.stdout:637 trunc from 0xbdbba16 to 0xee4bb0d 2024-09-18T01:07:54.102 INFO:teuthology.orchestra.run.smithi120.stdout:639 read 0xd7a7700 thru 0xd7b18ac (0xa1ad bytes) 2024-09-18T01:07:54.102 INFO:teuthology.orchestra.run.smithi120.stdout:640 read 0x135e9ca thru 0x1366794 (0x7dcb bytes) 2024-09-18T01:07:54.102 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.102 INFO:teuthology.orchestra.run.smithi120.stdout:642 read 0x701824a thru 0x701daa1 (0x5858 bytes) 2024-09-18T01:07:54.102 INFO:teuthology.orchestra.run.smithi120.stdout:643 writesame 0xcd6e77f thru 0xcd7570d data_size 0x1(0x6f8f bytes) 2024-09-18T01:07:54.122 INFO:teuthology.orchestra.run.smithi120.stdout:644 trunc from 0xee4bb0d to 0xa641100 2024-09-18T01:07:54.135 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:54.134+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:54.138 INFO:teuthology.orchestra.run.smithi120.stdout:645 writesame 0x7dd3610 thru 0x7de22f5 data_size 0x1(0xece6 bytes) 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.214646561999999, 2024-09-18T01:07:54.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:54.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:07:54.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:783 3.0 e28/23 3:04cd6778:::rbd_data.11769efc726f.0000000000000005:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.108455436, 2024-09-18T01:07:54.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026120100000000001, 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045917+0000", 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0500000000000003e-07 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046106+0000", 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018893600000000001 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046125+0000", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9128e-05 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046137+0000", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046199+0000", 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2520000000000004e-05 2024-09-18T01:07:54.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046222+0000", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2554000000000001e-05 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048480+0000", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002258047 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048528+0000", 2024-09-18T01:07:54.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8241000000000001e-05 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.086718522000002, 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:07:54.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1145 3.0 e28/23 3:0368a151:::rbd_data.11769efc726f.000000000000001b:head v 28'7, mlcod=28'5)", 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.062882491, 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026055610000000002, 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091489+0000", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091490+0000", 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:07:54.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091544+0000", 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3934999999999998e-05 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091563+0000", 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8423e-05 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091576+0000", 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3010000000000001e-05 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091636+0000", 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0732000000000001e-05 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.091658+0000", 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1943999999999999e-05 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094060+0000", 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00240136 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.094095+0000", 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.175 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.036344461999999, 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:07:54.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1568 3.9 e28/23 3:92680351:::rbd_data.11769efc726f.000000000000002b:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.027770083, 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026420340000000001, 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.180 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126603+0000", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.215e-06 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126978+0000", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037491299999999999 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126997+0000", 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9375e-05 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127007+0000", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127067+0000", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9812000000000003e-05 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127090+0000", 2024-09-18T01:07:54.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3828e-05 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129205+0000", 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002114215 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129243+0000", 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8751000000000002e-05 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2196 3.4 e28/23 3:26396459:::rbd_data.11769efc726f.000000000000003e:head v 28'7, mlcod=26'5)", 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.948025696999999, 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028009300000000001, 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206347+0000", 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.341e-06 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206440+0000", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3080999999999999e-05 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206458+0000", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7555999999999999e-05 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206467+0000", 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1519999999999998e-06 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206543+0000", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5899e-05 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206565+0000", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2497000000000002e-05 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209098+0000", 2024-09-18T01:07:54.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025328080000000001 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209147+0000", 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8596e-05 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.946451245999999, 2024-09-18T01:07:54.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:07:54.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.944855497999999, 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:07:54.194 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:07:54.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.944044879, 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.199 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:07:54.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.893497386, 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:54.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:07:54.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:07:54.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:3134 3.4 e29/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 29'16, mlcod=28'13)", 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.946123450999998, 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031975599999999999, 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:07:54.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208255+0000", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.319e-06 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208262+0000", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4309999999999997e-06 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208267+0000", 2024-09-18T01:07:54.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0370000000000002e-06 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208575+0000", 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030814599999999997 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208594+0000", 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8635999999999999e-05 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211406+0000", 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028115670000000001 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211446+0000", 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0021999999999997e-05 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:07:54.213 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.560672068000001, 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:07:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 152 entries 38'73 (0'0) error 3:07252d6d:::rbd_data.117611b60058.00000000000000af:head by client.4470.0:13143 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'71)", 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.129491+0000", 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.0248799779999995, 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022638269999999999, 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129491+0000", 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129491+0000", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129492+0000", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129500+0000", 2024-09-18T01:07:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2600000000000005e-06 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129521+0000", 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0727999999999999e-05 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129536+0000", 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4633e-05 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.129625+0000", 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9065000000000002e-05 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.131755+0000", 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002130481 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:07:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9970720110000002, 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:07:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 162 entries 38'77 (0'0) error 3:0796c97e:::rbd_data.117611b60058.0000000000000155:head by client.4470.0:13249 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'75)", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9803380239999999, 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030461970000000001, 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174034+0000", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174041+0000", 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:07:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174060+0000", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8848999999999999e-05 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174069+0000", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8950000000000004e-06 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174139+0000", 2024-09-18T01:07:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0235999999999996e-05 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177080+0000", 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029401760000000001 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9793109250000001, 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9766542220000005, 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:07:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:07:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:15261 3.0 e43/23 3:03957b90:::rbd_data.1176492bc9ca.000000000000000c:head v 43'81, mlcod=40'79)", 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.563955, 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031977450000000001, 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590418+0000", 2024-09-18T01:07:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4890000000000001e-06 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590594+0000", 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017577099999999999 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590610+0000", 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6327000000000001e-05 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590618+0000", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3839999999999997e-06 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590682+0000", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3596000000000006e-05 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590708+0000", 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6089e-05 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593568+0000", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028597789999999998 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593614+0000", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-05 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.39400667900000003, 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:07:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:07:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:54.248 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:54.249 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:54.250 INFO:teuthology.orchestra.run.smithi120.stdout:646 write 0xa829746 thru 0xa837003 (0xd8be bytes) 2024-09-18T01:07:54.250 INFO:teuthology.orchestra.run.smithi120.stdout:647 write 0x2514805 thru 0x251f608 (0xae04 bytes) 2024-09-18T01:07:54.250 INFO:teuthology.orchestra.run.smithi120.stdout:648 write 0xa08147 thru 0xa0d1bf (0x5079 bytes) 2024-09-18T01:07:54.250 INFO:teuthology.orchestra.run.smithi120.stdout:649 write 0xd4fe47 thru 0xd5dd29 (0xdee3 bytes) 2024-09-18T01:07:54.250 INFO:teuthology.orchestra.run.smithi120.stdout:650 punch from 0x9b737a4 to 0x9b7738c, (0x3be8 bytes) 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:651 read 0x2f97b4c thru 0x2f9cdd7 (0x528c bytes) 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:654 read 0x996bde8 thru 0x9972cc1 (0x6eda bytes) 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:655 writesame 0xe98d172 thru 0xe9956b3 data_size 0x25(0x8542 bytes) 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:54.251 INFO:teuthology.orchestra.run.smithi120.stdout:659 clone 8 order 20 su 8192 sc 13 2024-09-18T01:07:54.251 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:54.248+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:54.340 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:54.349 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:54.349+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:54.395 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:54.395 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:54.395 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:54.395 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:54.403 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:54.450 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:54.449+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:54.461 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:54.461 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:54.478 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:54.478 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:54.496 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:54.510 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:54.510 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:54.510 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:54.510 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:54.510 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:54.512 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:54.513 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:54.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:54.529 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:07:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:07:54.551 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:54.550+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:54.554 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:54.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:07:54.573 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:54.593 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:54.593 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:54.648 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:54.688 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:54.687+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:54.751 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:54.750+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:54.752 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:54.752 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:54.761 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:54.852 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:54.851+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:54.904 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:54.905 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:54.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:54.957 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:54.956+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:55.017 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.017 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.026 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:55.054 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:55.053+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:55.155 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:55.154+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:55.171 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.171 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:55.256 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:55.256+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:55.325 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.325 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.334 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:55.357 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:55.356+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:55.437 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.437 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.445 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:55.458 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:55.457+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:07:55.553 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.554 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.559 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:55.558+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:55.561 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:07:55.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:55.566 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:55.662 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:55.659+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:55.703 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.703 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:55.761 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:55.760+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:55.854 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:55.854 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:55.862 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:55.861+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:55.866 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:55.962 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:55.961+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:56.016 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.016 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.025 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:56.056 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:56.063 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:56.062+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:56.068 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-18T01:07:55.540021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328767764,"num_objects":319,"num_object_clones":0,"num_object_copies":638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":18757,"num_read_kb":294109,"num_write":3468,"num_write_kb":192210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2722,"ondisk_log_size":2722,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479944,"kb_used_data":238748,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754494776,"statfs":{"total":773094113280,"available":772602650624,"internally_reserved":0,"allocated":244477952,"data_stored":241305248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26091121,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":2885,"num_read_kb":21930,"num_write":562,"num_write_kb":14059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.947502"},"pg_stats":[{"pgid":"3.1f","version":"46'104","reported_seq":730,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.612627+0000","last_change":"2024-09-18T01:07:50.594798+0000","last_active":"2024-09-18T01:07:52.612627+0000","last_peered":"2024-09-18T01:07:52.612627+0000","last_clean":"2024-09-18T01:07:52.612627+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:07:52.612627+0000","last_undegraded":"2024-09-18T01:07:52.612627+0000","last_fullsized":"2024-09-18T01:07:52.612627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:09.247567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011810970000000001,"stat_sum":{"num_bytes":16708952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":788,"num_read_kb":10972,"num_write":127,"num_write_kb":13315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1e","version":"44'63","reported_seq":369,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.554745+0000","last_change":"2024-09-18T01:07:53.640795+0000","last_active":"2024-09-18T01:07:54.554745+0000","last_peered":"2024-09-18T01:07:54.554745+0000","last_clean":"2024-09-18T01:07:54.554745+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:07:54.554745+0000","last_undegraded":"2024-09-18T01:07:54.554745+0000","last_fullsized":"2024-09-18T01:07:54.554745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:54.790773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056291900000000005,"stat_sum":{"num_bytes":7322461,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":296,"num_read_kb":9541,"num_write":74,"num_write_kb":4679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.1d","version":"46'70","reported_seq":679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613969+0000","last_change":"2024-09-18T01:07:50.600276+0000","last_active":"2024-09-18T01:07:52.613969+0000","last_peered":"2024-09-18T01:07:52.613969+0000","last_clean":"2024-09-18T01:07:52.613969+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:07:52.613969+0000","last_undegraded":"2024-09-18T01:07:52.613969+0000","last_fullsized":"2024-09-18T01:07:52.613969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:29:55.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076780700000000002,"stat_sum":{"num_bytes":16221348,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":11167,"num_write":64,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1c","version":"46'36","reported_seq":250,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.653890+0000","last_change":"2024-09-18T01:07:50.599983+0000","last_active":"2024-09-18T01:07:52.653890+0000","last_peered":"2024-09-18T01:07:52.653890+0000","last_clean":"2024-09-18T01:07:52.653890+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:07:52.653890+0000","last_undegraded":"2024-09-18T01:07:52.653890+0000","last_fullsized":"2024-09-18T01:07:52.653890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:00:51.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000700159,"stat_sum":{"num_bytes":4927667,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":129,"num_read_kb":5178,"num_write":21,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1b","version":"46'202","reported_seq":987,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.720472+0000","last_change":"2024-09-18T01:07:50.594366+0000","last_active":"2024-09-18T01:07:52.720472+0000","last_peered":"2024-09-18T01:07:52.720472+0000","last_clean":"2024-09-18T01:07:52.720472+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:07:52.720472+0000","last_undegraded":"2024-09-18T01:07:52.720472+0000","last_fullsized":"2024-09-18T01:07:52.720472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:26.704131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091815500000000003,"stat_sum":{"num_bytes":20287898,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1424,"num_read_kb":16624,"num_write":274,"num_write_kb":10216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1a","version":"40'67","reported_seq":730,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.722726+0000","last_change":"2024-09-18T01:07:50.593459+0000","last_active":"2024-09-18T01:07:50.722726+0000","last_peered":"2024-09-18T01:07:50.722726+0000","last_clean":"2024-09-18T01:07:50.722726+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:07:50.722726+0000","last_undegraded":"2024-09-18T01:07:50.722726+0000","last_fullsized":"2024-09-18T01:07:50.722726+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:41.759601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039271999999999999,"stat_sum":{"num_bytes":12409280,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":872,"num_read_kb":14037,"num_write":67,"num_write_kb":6929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.19","version":"46'49","reported_seq":751,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.674570+0000","last_change":"2024-09-18T01:07:50.672994+0000","last_active":"2024-09-18T01:07:52.674570+0000","last_peered":"2024-09-18T01:07:52.674570+0000","last_clean":"2024-09-18T01:07:52.674570+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:07:52.674570+0000","last_undegraded":"2024-09-18T01:07:52.674570+0000","last_fullsized":"2024-09-18T01:07:52.674570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:32:39.008201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034410700000000002,"stat_sum":{"num_bytes":18041747,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":599,"num_read_kb":13159,"num_write":50,"num_write_kb":8283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.18","version":"48'35","reported_seq":489,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.559543+0000","last_change":"2024-09-18T01:07:53.640528+0000","last_active":"2024-09-18T01:07:54.559543+0000","last_peered":"2024-09-18T01:07:54.559543+0000","last_clean":"2024-09-18T01:07:54.559543+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:07:54.559543+0000","last_undegraded":"2024-09-18T01:07:54.559543+0000","last_fullsized":"2024-09-18T01:07:54.559543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:56.003945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035945600000000002,"stat_sum":{"num_bytes":11018430,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":344,"num_read_kb":4084,"num_write":21,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.17","version":"46'80","reported_seq":466,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.659245+0000","last_change":"2024-09-18T01:07:50.594898+0000","last_active":"2024-09-18T01:07:52.659245+0000","last_peered":"2024-09-18T01:07:52.659245+0000","last_clean":"2024-09-18T01:07:52.659245+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:07:52.659245+0000","last_undegraded":"2024-09-18T01:07:52.659245+0000","last_fullsized":"2024-09-18T01:07:52.659245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:01.506210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793109999999999,"stat_sum":{"num_bytes":10083100,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":448,"num_read_kb":11343,"num_write":96,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.16","version":"46'29","reported_seq":486,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.676167+0000","last_change":"2024-09-18T01:07:50.599680+0000","last_active":"2024-09-18T01:07:52.676167+0000","last_peered":"2024-09-18T01:07:52.676167+0000","last_clean":"2024-09-18T01:07:52.676167+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:07:52.676167+0000","last_undegraded":"2024-09-18T01:07:52.676167+0000","last_fullsized":"2024-09-18T01:07:52.676167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:38:31.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044660499999999998,"stat_sum":{"num_bytes":7917046,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":356,"num_read_kb":9320,"num_write":21,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.15","version":"46'69","reported_seq":863,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.652308+0000","last_change":"2024-09-18T01:07:50.674379+0000","last_active":"2024-09-18T01:07:52.652308+0000","last_peered":"2024-09-18T01:07:52.652308+0000","last_clean":"2024-09-18T01:07:52.652308+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:07:52.652308+0000","last_undegraded":"2024-09-18T01:07:52.652308+0000","last_fullsized":"2024-09-18T01:07:52.652308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:34.645334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066622399999999998,"stat_sum":{"num_bytes":12775292,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":894,"num_read_kb":14951,"num_write":62,"num_write_kb":5701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.14","version":"43'99","reported_seq":335,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593554+0000","last_change":"2024-09-18T01:07:50.593760+0000","last_active":"2024-09-18T01:07:50.593554+0000","last_peered":"2024-09-18T01:07:50.593554+0000","last_clean":"2024-09-18T01:07:50.593554+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:07:50.593554+0000","last_undegraded":"2024-09-18T01:07:50.593554+0000","last_fullsized":"2024-09-18T01:07:50.593554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:27.797234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046991300000000003,"stat_sum":{"num_bytes":3850305,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":499,"num_read_kb":4048,"num_write":139,"num_write_kb":2400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.624335+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:52.624335+0000","last_peered":"2024-09-18T01:07:52.624335+0000","last_clean":"2024-09-18T01:07:52.624335+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:52.624335+0000","last_undegraded":"2024-09-18T01:07:52.624335+0000","last_fullsized":"2024-09-18T01:07:52.624335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"46'45","reported_seq":492,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.612671+0000","last_change":"2024-09-18T01:07:50.593774+0000","last_active":"2024-09-18T01:07:52.612671+0000","last_peered":"2024-09-18T01:07:52.612671+0000","last_clean":"2024-09-18T01:07:52.612671+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:07:52.612671+0000","last_undegraded":"2024-09-18T01:07:52.612671+0000","last_fullsized":"2024-09-18T01:07:52.612671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:37.834448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042427900000000001,"stat_sum":{"num_bytes":4319928,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":655,"num_read_kb":3853,"num_write":37,"num_write_kb":628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.620251+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:52.620251+0000","last_peered":"2024-09-18T01:07:52.620251+0000","last_clean":"2024-09-18T01:07:52.620251+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:52.620251+0000","last_undegraded":"2024-09-18T01:07:52.620251+0000","last_fullsized":"2024-09-18T01:07:52.620251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'166","reported_seq":497,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555026+0000","last_change":"2024-09-18T01:07:53.640595+0000","last_active":"2024-09-18T01:07:54.555026+0000","last_peered":"2024-09-18T01:07:54.555026+0000","last_clean":"2024-09-18T01:07:54.555026+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:07:54.555026+0000","last_undegraded":"2024-09-18T01:07:54.555026+0000","last_fullsized":"2024-09-18T01:07:54.555026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:13.403023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038714700000000001,"stat_sum":{"num_bytes":7258112,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":822,"num_read_kb":10145,"num_write":282,"num_write_kb":8577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611934+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:52.611934+0000","last_peered":"2024-09-18T01:07:52.611934+0000","last_clean":"2024-09-18T01:07:52.611934+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:52.611934+0000","last_undegraded":"2024-09-18T01:07:52.611934+0000","last_fullsized":"2024-09-18T01:07:52.611934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"46'75","reported_seq":1062,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.632607+0000","last_change":"2024-09-18T01:07:50.673757+0000","last_active":"2024-09-18T01:07:52.632607+0000","last_peered":"2024-09-18T01:07:52.632607+0000","last_clean":"2024-09-18T01:07:52.632607+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:07:52.632607+0000","last_undegraded":"2024-09-18T01:07:52.632607+0000","last_fullsized":"2024-09-18T01:07:52.632607+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:45:45.418410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565780000000001,"stat_sum":{"num_bytes":19066141,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1014,"num_read_kb":22595,"num_write":80,"num_write_kb":7465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.620222+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:52.620222+0000","last_peered":"2024-09-18T01:07:52.620222+0000","last_clean":"2024-09-18T01:07:52.620222+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:52.620222+0000","last_undegraded":"2024-09-18T01:07:52.620222+0000","last_fullsized":"2024-09-18T01:07:52.620222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'43","reported_seq":715,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613876+0000","last_change":"2024-09-18T01:07:50.599672+0000","last_active":"2024-09-18T01:07:52.613876+0000","last_peered":"2024-09-18T01:07:52.613876+0000","last_clean":"2024-09-18T01:07:52.613876+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:07:52.613876+0000","last_undegraded":"2024-09-18T01:07:52.613876+0000","last_fullsized":"2024-09-18T01:07:52.613876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:33:16.474272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040002599999999999,"stat_sum":{"num_bytes":22988662,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":582,"num_read_kb":10604,"num_write":41,"num_write_kb":8952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.3","version":"43'74","reported_seq":582,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593517+0000","last_change":"2024-09-18T01:07:50.593768+0000","last_active":"2024-09-18T01:07:50.593517+0000","last_peered":"2024-09-18T01:07:50.593517+0000","last_clean":"2024-09-18T01:07:50.593517+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:07:50.593517+0000","last_undegraded":"2024-09-18T01:07:50.593517+0000","last_fullsized":"2024-09-18T01:07:50.593517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:23.958763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622472,"stat_sum":{"num_bytes":5914624,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":859,"num_read_kb":7864,"num_write":74,"num_write_kb":3507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613538+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:52.613538+0000","last_peered":"2024-09-18T01:07:52.613538+0000","last_clean":"2024-09-18T01:07:52.613538+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:52.613538+0000","last_undegraded":"2024-09-18T01:07:52.613538+0000","last_fullsized":"2024-09-18T01:07:52.613538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"43'81","reported_seq":740,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.652247+0000","last_change":"2024-09-18T01:07:50.673042+0000","last_active":"2024-09-18T01:07:52.652247+0000","last_peered":"2024-09-18T01:07:52.652247+0000","last_clean":"2024-09-18T01:07:52.652247+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:07:52.652247+0000","last_undegraded":"2024-09-18T01:07:52.652247+0000","last_fullsized":"2024-09-18T01:07:52.652247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:17.778103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366568,"stat_sum":{"num_bytes":15777539,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":811,"num_read_kb":20355,"num_write":137,"num_write_kb":8749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611667+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:52.611667+0000","last_peered":"2024-09-18T01:07:52.611667+0000","last_clean":"2024-09-18T01:07:52.611667+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:52.611667+0000","last_undegraded":"2024-09-18T01:07:52.611667+0000","last_fullsized":"2024-09-18T01:07:52.611667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'110","reported_seq":934,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.669676+0000","last_change":"2024-09-18T01:07:50.594666+0000","last_active":"2024-09-18T01:07:52.669676+0000","last_peered":"2024-09-18T01:07:52.669676+0000","last_clean":"2024-09-18T01:07:52.669676+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:07:52.669676+0000","last_undegraded":"2024-09-18T01:07:52.669676+0000","last_fullsized":"2024-09-18T01:07:52.669676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:26.969625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079864,"stat_sum":{"num_bytes":25597834,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":973,"num_read_kb":8239,"num_write":104,"num_write_kb":13556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611623+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:52.611623+0000","last_peered":"2024-09-18T01:07:52.611623+0000","last_clean":"2024-09-18T01:07:52.611623+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:52.611623+0000","last_undegraded":"2024-09-18T01:07:52.611623+0000","last_fullsized":"2024-09-18T01:07:52.611623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"48'79","reported_seq":676,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.594047+0000","last_change":"2024-09-18T01:07:53.641045+0000","last_active":"2024-09-18T01:07:54.594047+0000","last_peered":"2024-09-18T01:07:54.594047+0000","last_clean":"2024-09-18T01:07:54.594047+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:07:54.594047+0000","last_undegraded":"2024-09-18T01:07:54.594047+0000","last_fullsized":"2024-09-18T01:07:54.594047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:16.106450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073822199999999997,"stat_sum":{"num_bytes":16991697,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":518,"num_read_kb":4915,"num_write":74,"num_write_kb":12399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613530+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:52.613530+0000","last_peered":"2024-09-18T01:07:52.613530+0000","last_clean":"2024-09-18T01:07:52.613530+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:52.613530+0000","last_undegraded":"2024-09-18T01:07:52.613530+0000","last_fullsized":"2024-09-18T01:07:52.613530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611634+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:52.611634+0000","last_peered":"2024-09-18T01:07:52.611634+0000","last_clean":"2024-09-18T01:07:52.611634+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:52.611634+0000","last_undegraded":"2024-09-18T01:07:52.611634+0000","last_fullsized":"2024-09-18T01:07:52.611634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"47'34","reported_seq":307,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555197+0000","last_change":"2024-09-18T01:07:53.641041+0000","last_active":"2024-09-18T01:07:54.555197+0000","last_peered":"2024-09-18T01:07:54.555197+0000","last_clean":"2024-09-18T01:07:54.555197+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:07:54.555197+0000","last_undegraded":"2024-09-18T01:07:54.555197+0000","last_fullsized":"2024-09-18T01:07:54.555197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:20.348592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679694,"stat_sum":{"num_bytes":3981328,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":5099,"num_write":27,"num_write_kb":1804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.9","version":"38'30","reported_seq":320,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593461+0000","last_change":"2024-09-18T01:07:50.593461+0000","last_active":"2024-09-18T01:07:50.593461+0000","last_peered":"2024-09-18T01:07:50.593461+0000","last_clean":"2024-09-18T01:07:50.593461+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:07:50.593461+0000","last_undegraded":"2024-09-18T01:07:50.593461+0000","last_fullsized":"2024-09-18T01:07:50.593461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:17:31.135951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034929399999999999,"stat_sum":{"num_bytes":7527681,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":193,"num_read_kb":12121,"num_write":23,"num_write_kb":5474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.a","version":"46'31","reported_seq":297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.624474+0000","last_change":"2024-09-18T01:07:50.673650+0000","last_active":"2024-09-18T01:07:52.624474+0000","last_peered":"2024-09-18T01:07:52.624474+0000","last_clean":"2024-09-18T01:07:52.624474+0000","last_became_active":"2024-09-18T01:07:28.974981+0000","last_became_peered":"2024-09-18T01:07:28.974981+0000","last_unstale":"2024-09-18T01:07:52.624474+0000","last_undegraded":"2024-09-18T01:07:52.624474+0000","last_fullsized":"2024-09-18T01:07:52.624474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:03.020699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046493499999999998,"stat_sum":{"num_bytes":4247889,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":165,"num_read_kb":5376,"num_write":26,"num_write_kb":2894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.b","version":"47'62","reported_seq":580,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555469+0000","last_change":"2024-09-18T01:07:53.641181+0000","last_active":"2024-09-18T01:07:54.555469+0000","last_peered":"2024-09-18T01:07:54.555469+0000","last_clean":"2024-09-18T01:07:54.555469+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:07:54.555469+0000","last_undegraded":"2024-09-18T01:07:54.555469+0000","last_fullsized":"2024-09-18T01:07:54.555469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:20.405849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277499999999999,"stat_sum":{"num_bytes":10812299,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":509,"num_read_kb":14498,"num_write":78,"num_write_kb":5730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.c","version":"46'409","reported_seq":749,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.634590+0000","last_change":"2024-09-18T01:07:50.600021+0000","last_active":"2024-09-18T01:07:52.634590+0000","last_peered":"2024-09-18T01:07:52.634590+0000","last_clean":"2024-09-18T01:07:52.634590+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:07:52.634590+0000","last_undegraded":"2024-09-18T01:07:52.634590+0000","last_fullsized":"2024-09-18T01:07:52.634590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:08:38.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072939299999999995,"stat_sum":{"num_bytes":4696632,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2163,"num_read_kb":5694,"num_write":784,"num_write_kb":1716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.d","version":"46'61","reported_seq":436,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.721887+0000","last_change":"2024-09-18T01:07:50.674706+0000","last_active":"2024-09-18T01:07:52.721887+0000","last_peered":"2024-09-18T01:07:52.721887+0000","last_clean":"2024-09-18T01:07:52.721887+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:07:52.721887+0000","last_undegraded":"2024-09-18T01:07:52.721887+0000","last_fullsized":"2024-09-18T01:07:52.721887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:38:39.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072260800000000002,"stat_sum":{"num_bytes":8752607,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":346,"num_read_kb":9466,"num_write":51,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.e","version":"46'128","reported_seq":491,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.673879+0000","last_change":"2024-09-18T01:07:50.674842+0000","last_active":"2024-09-18T01:07:52.673879+0000","last_peered":"2024-09-18T01:07:52.673879+0000","last_clean":"2024-09-18T01:07:52.673879+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:07:52.673879+0000","last_undegraded":"2024-09-18T01:07:52.673879+0000","last_fullsized":"2024-09-18T01:07:52.673879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:48.391131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098800000000002,"stat_sum":{"num_bytes":10032599,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":772,"num_read_kb":10945,"num_write":223,"num_write_kb":6093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.f","version":"46'29","reported_seq":265,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.676003+0000","last_change":"2024-09-18T01:07:50.674619+0000","last_active":"2024-09-18T01:07:52.676003+0000","last_peered":"2024-09-18T01:07:52.676003+0000","last_clean":"2024-09-18T01:07:52.676003+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:07:52.676003+0000","last_undegraded":"2024-09-18T01:07:52.676003+0000","last_fullsized":"2024-09-18T01:07:52.676003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:15.729758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066955400000000005,"stat_sum":{"num_bytes":5769900,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":133,"num_read_kb":5056,"num_write":16,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.10","version":"46'62","reported_seq":407,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.661758+0000","last_change":"2024-09-18T01:07:50.673690+0000","last_active":"2024-09-18T01:07:52.661758+0000","last_peered":"2024-09-18T01:07:52.661758+0000","last_clean":"2024-09-18T01:07:52.661758+0000","last_became_active":"2024-09-18T01:07:28.976873+0000","last_became_peered":"2024-09-18T01:07:28.976873+0000","last_unstale":"2024-09-18T01:07:52.661758+0000","last_undegraded":"2024-09-18T01:07:52.661758+0000","last_fullsized":"2024-09-18T01:07:52.661758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:29.577702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045779700000000002,"stat_sum":{"num_bytes":8438077,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":310,"num_read_kb":6314,"num_write":59,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.11","version":"46'63","reported_seq":177,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.722347+0000","last_change":"2024-09-18T01:07:50.674145+0000","last_active":"2024-09-18T01:07:52.722347+0000","last_peered":"2024-09-18T01:07:52.722347+0000","last_clean":"2024-09-18T01:07:52.722347+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:07:52.722347+0000","last_undegraded":"2024-09-18T01:07:52.722347+0000","last_fullsized":"2024-09-18T01:07:52.722347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:26:41.996780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044894299999999999,"stat_sum":{"num_bytes":262144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":203,"num_read_kb":922,"num_write":91,"num_write_kb":559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.12","version":"46'79","reported_seq":317,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.630648+0000","last_change":"2024-09-18T01:07:50.594433+0000","last_active":"2024-09-18T01:07:52.630648+0000","last_peered":"2024-09-18T01:07:52.630648+0000","last_clean":"2024-09-18T01:07:52.630648+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:07:52.630648+0000","last_undegraded":"2024-09-18T01:07:52.630648+0000","last_fullsized":"2024-09-18T01:07:52.630648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:08.614064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093869099999999996,"stat_sum":{"num_bytes":3123076,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":233,"num_read_kb":3429,"num_write":63,"num_write_kb":2037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.13","version":"44'38","reported_seq":195,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.632614+0000","last_change":"2024-09-18T01:07:50.674463+0000","last_active":"2024-09-18T01:07:52.632614+0000","last_peered":"2024-09-18T01:07:52.632614+0000","last_clean":"2024-09-18T01:07:52.632614+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:07:52.632614+0000","last_undegraded":"2024-09-18T01:07:52.632614+0000","last_fullsized":"2024-09-18T01:07:52.632614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:55.753549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848800000000001,"stat_sum":{"num_bytes":1055082,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":54,"num_read_kb":2094,"num_write":21,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":328177377,"num_objects":315,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":315,"num_whiteouts":0,"num_read":18640,"num_read_kb":294008,"num_write":3307,"num_write_kb":189792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":248483840,"data_stored":247611507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2602,"ondisk_log_size":2602,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64148,"kb_used_data":32964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307692,"statfs":{"total":96636764160,"available":96571076608,"internally_reserved":0,"allocated":33755136,"data_stored":33396612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":9296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":9519104,"data_stored":9202353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62436,"kb_used_data":31248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309404,"statfs":{"total":96636764160,"available":96572829696,"internally_reserved":0,"allocated":31997952,"data_stored":31583815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47988,"kb_used_data":20960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323852,"statfs":{"total":96636764160,"available":96587624448,"internally_reserved":0,"allocated":21463040,"data_stored":21108591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59544,"kb_used_data":28356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312296,"statfs":{"total":96636764160,"available":96575791104,"internally_reserved":0,"allocated":29036544,"data_stored":28619512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65672,"kb_used_data":34484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306168,"statfs":{"total":96636764160,"available":96569516032,"internally_reserved":0,"allocated":35311616,"data_stored":34890339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74516,"kb_used_data":43324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297324,"statfs":{"total":96636764160,"available":96560459776,"internally_reserved":0,"allocated":44363776,"data_stored":43966716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69312,"kb_used_data":38116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302528,"statfs":{"total":96636764160,"available":96565788672,"internally_reserved":0,"allocated":39030784,"data_stored":38537310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37265408,"data_stored":37123191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29220864,"data_stored":29089429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21516288,"data_stored":21446726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":38490112,"data_stored":38299708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43753974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35512320,"data_stored":35383760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9965568,"data_stored":9933704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32581015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:56.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:07:56.163 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:56.163+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:56.168 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.168 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.177 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:56.264 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:56.263+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:56.280 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.280 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:07:56.365 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:56.365+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:56.389 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:07:56.391 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:07:56.393 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.393 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.402 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-18T01:07:55.540021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328767764,"num_objects":319,"num_object_clones":0,"num_object_copies":638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":18757,"num_read_kb":294109,"num_write":3468,"num_write_kb":192210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2722,"ondisk_log_size":2722,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479944,"kb_used_data":238748,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754494776,"statfs":{"total":773094113280,"available":772602650624,"internally_reserved":0,"allocated":244477952,"data_stored":241305248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26091121,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":2885,"num_read_kb":21930,"num_write":562,"num_write_kb":14059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.947502"},"pg_stats":[{"pgid":"3.1f","version":"46'104","reported_seq":730,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.612627+0000","last_change":"2024-09-18T01:07:50.594798+0000","last_active":"2024-09-18T01:07:52.612627+0000","last_peered":"2024-09-18T01:07:52.612627+0000","last_clean":"2024-09-18T01:07:52.612627+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:07:52.612627+0000","last_undegraded":"2024-09-18T01:07:52.612627+0000","last_fullsized":"2024-09-18T01:07:52.612627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:09.247567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011810970000000001,"stat_sum":{"num_bytes":16708952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":788,"num_read_kb":10972,"num_write":127,"num_write_kb":13315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1e","version":"44'63","reported_seq":369,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.554745+0000","last_change":"2024-09-18T01:07:53.640795+0000","last_active":"2024-09-18T01:07:54.554745+0000","last_peered":"2024-09-18T01:07:54.554745+0000","last_clean":"2024-09-18T01:07:54.554745+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:07:54.554745+0000","last_undegraded":"2024-09-18T01:07:54.554745+0000","last_fullsized":"2024-09-18T01:07:54.554745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:54.790773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056291900000000005,"stat_sum":{"num_bytes":7322461,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":296,"num_read_kb":9541,"num_write":74,"num_write_kb":4679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.1d","version":"46'70","reported_seq":679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613969+0000","last_change":"2024-09-18T01:07:50.600276+0000","last_active":"2024-09-18T01:07:52.613969+0000","last_peered":"2024-09-18T01:07:52.613969+0000","last_clean":"2024-09-18T01:07:52.613969+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:07:52.613969+0000","last_undegraded":"2024-09-18T01:07:52.613969+0000","last_fullsized":"2024-09-18T01:07:52.613969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:29:55.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076780700000000002,"stat_sum":{"num_bytes":16221348,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":11167,"num_write":64,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1c","version":"46'36","reported_seq":250,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.653890+0000","last_change":"2024-09-18T01:07:50.599983+0000","last_active":"2024-09-18T01:07:52.653890+0000","last_peered":"2024-09-18T01:07:52.653890+0000","last_clean":"2024-09-18T01:07:52.653890+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:07:52.653890+0000","last_undegraded":"2024-09-18T01:07:52.653890+0000","last_fullsized":"2024-09-18T01:07:52.653890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:00:51.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000700159,"stat_sum":{"num_bytes":4927667,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":129,"num_read_kb":5178,"num_write":21,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1b","version":"46'202","reported_seq":987,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.720472+0000","last_change":"2024-09-18T01:07:50.594366+0000","last_active":"2024-09-18T01:07:52.720472+0000","last_peered":"2024-09-18T01:07:52.720472+0000","last_clean":"2024-09-18T01:07:52.720472+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:07:52.720472+0000","last_undegraded":"2024-09-18T01:07:52.720472+0000","last_fullsized":"2024-09-18T01:07:52.720472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:26.704131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091815500000000003,"stat_sum":{"num_bytes":20287898,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1424,"num_read_kb":16624,"num_write":274,"num_write_kb":10216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.1a","version":"40'67","reported_seq":730,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.722726+0000","last_change":"2024-09-18T01:07:50.593459+0000","last_active":"2024-09-18T01:07:50.722726+0000","last_peered":"2024-09-18T01:07:50.722726+0000","last_clean":"2024-09-18T01:07:50.722726+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:07:50.722726+0000","last_undegraded":"2024-09-18T01:07:50.722726+0000","last_fullsized":"2024-09-18T01:07:50.722726+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:41.759601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039271999999999999,"stat_sum":{"num_bytes":12409280,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":872,"num_read_kb":14037,"num_write":67,"num_write_kb":6929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.19","version":"46'49","reported_seq":751,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.674570+0000","last_change":"2024-09-18T01:07:50.672994+0000","last_active":"2024-09-18T01:07:52.674570+0000","last_peered":"2024-09-18T01:07:52.674570+0000","last_clean":"2024-09-18T01:07:52.674570+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:07:52.674570+0000","last_undegraded":"2024-09-18T01:07:52.674570+0000","last_fullsized":"2024-09-18T01:07:52.674570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:32:39.008201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034410700000000002,"stat_sum":{"num_bytes":18041747,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":599,"num_read_kb":13159,"num_write":50,"num_write_kb":8283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.18","version":"48'35","reported_seq":489,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.559543+0000","last_change":"2024-09-18T01:07:53.640528+0000","last_active":"2024-09-18T01:07:54.559543+0000","last_peered":"2024-09-18T01:07:54.559543+0000","last_clean":"2024-09-18T01:07:54.559543+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:07:54.559543+0000","last_undegraded":"2024-09-18T01:07:54.559543+0000","last_fullsized":"2024-09-18T01:07:54.559543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:56.003945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035945600000000002,"stat_sum":{"num_bytes":11018430,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":344,"num_read_kb":4084,"num_write":21,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.17","version":"46'80","reported_seq":466,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.659245+0000","last_change":"2024-09-18T01:07:50.594898+0000","last_active":"2024-09-18T01:07:52.659245+0000","last_peered":"2024-09-18T01:07:52.659245+0000","last_clean":"2024-09-18T01:07:52.659245+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:07:52.659245+0000","last_undegraded":"2024-09-18T01:07:52.659245+0000","last_fullsized":"2024-09-18T01:07:52.659245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:01.506210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793109999999999,"stat_sum":{"num_bytes":10083100,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":448,"num_read_kb":11343,"num_write":96,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.16","version":"46'29","reported_seq":486,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.676167+0000","last_change":"2024-09-18T01:07:50.599680+0000","last_active":"2024-09-18T01:07:52.676167+0000","last_peered":"2024-09-18T01:07:52.676167+0000","last_clean":"2024-09-18T01:07:52.676167+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:07:52.676167+0000","last_undegraded":"2024-09-18T01:07:52.676167+0000","last_fullsized":"2024-09-18T01:07:52.676167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:38:31.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044660499999999998,"stat_sum":{"num_bytes":7917046,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":356,"num_read_kb":9320,"num_write":21,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.15","version":"46'69","reported_seq":863,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.652308+0000","last_change":"2024-09-18T01:07:50.674379+0000","last_active":"2024-09-18T01:07:52.652308+0000","last_peered":"2024-09-18T01:07:52.652308+0000","last_clean":"2024-09-18T01:07:52.652308+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:07:52.652308+0000","last_undegraded":"2024-09-18T01:07:52.652308+0000","last_fullsized":"2024-09-18T01:07:52.652308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:34.645334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066622399999999998,"stat_sum":{"num_bytes":12775292,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":894,"num_read_kb":14951,"num_write":62,"num_write_kb":5701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.14","version":"43'99","reported_seq":335,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593554+0000","last_change":"2024-09-18T01:07:50.593760+0000","last_active":"2024-09-18T01:07:50.593554+0000","last_peered":"2024-09-18T01:07:50.593554+0000","last_clean":"2024-09-18T01:07:50.593554+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:07:50.593554+0000","last_undegraded":"2024-09-18T01:07:50.593554+0000","last_fullsized":"2024-09-18T01:07:50.593554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:27.797234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046991300000000003,"stat_sum":{"num_bytes":3850305,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":499,"num_read_kb":4048,"num_write":139,"num_write_kb":2400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.624335+0000","last_change":"2024-09-18T01:07:21.030442+0000","last_active":"2024-09-18T01:07:52.624335+0000","last_peered":"2024-09-18T01:07:52.624335+0000","last_clean":"2024-09-18T01:07:52.624335+0000","last_became_active":"2024-09-18T01:07:18.943453+0000","last_became_peered":"2024-09-18T01:07:18.943453+0000","last_unstale":"2024-09-18T01:07:52.624335+0000","last_undegraded":"2024-09-18T01:07:52.624335+0000","last_fullsized":"2024-09-18T01:07:52.624335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:17.174911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034387900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"46'45","reported_seq":492,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.612671+0000","last_change":"2024-09-18T01:07:50.593774+0000","last_active":"2024-09-18T01:07:52.612671+0000","last_peered":"2024-09-18T01:07:52.612671+0000","last_clean":"2024-09-18T01:07:52.612671+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:07:52.612671+0000","last_undegraded":"2024-09-18T01:07:52.612671+0000","last_fullsized":"2024-09-18T01:07:52.612671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:37.834448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042427900000000001,"stat_sum":{"num_bytes":4319928,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":655,"num_read_kb":3853,"num_write":37,"num_write_kb":628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.620251+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:07:52.620251+0000","last_peered":"2024-09-18T01:07:52.620251+0000","last_clean":"2024-09-18T01:07:52.620251+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:07:52.620251+0000","last_undegraded":"2024-09-18T01:07:52.620251+0000","last_fullsized":"2024-09-18T01:07:52.620251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'166","reported_seq":497,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555026+0000","last_change":"2024-09-18T01:07:53.640595+0000","last_active":"2024-09-18T01:07:54.555026+0000","last_peered":"2024-09-18T01:07:54.555026+0000","last_clean":"2024-09-18T01:07:54.555026+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:07:54.555026+0000","last_undegraded":"2024-09-18T01:07:54.555026+0000","last_fullsized":"2024-09-18T01:07:54.555026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:13.403023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038714700000000001,"stat_sum":{"num_bytes":7258112,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":822,"num_read_kb":10145,"num_write":282,"num_write_kb":8577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611934+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:07:52.611934+0000","last_peered":"2024-09-18T01:07:52.611934+0000","last_clean":"2024-09-18T01:07:52.611934+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:07:52.611934+0000","last_undegraded":"2024-09-18T01:07:52.611934+0000","last_fullsized":"2024-09-18T01:07:52.611934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"46'75","reported_seq":1062,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.632607+0000","last_change":"2024-09-18T01:07:50.673757+0000","last_active":"2024-09-18T01:07:52.632607+0000","last_peered":"2024-09-18T01:07:52.632607+0000","last_clean":"2024-09-18T01:07:52.632607+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:07:52.632607+0000","last_undegraded":"2024-09-18T01:07:52.632607+0000","last_fullsized":"2024-09-18T01:07:52.632607+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:45:45.418410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565780000000001,"stat_sum":{"num_bytes":19066141,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1014,"num_read_kb":22595,"num_write":80,"num_write_kb":7465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.620222+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:07:52.620222+0000","last_peered":"2024-09-18T01:07:52.620222+0000","last_clean":"2024-09-18T01:07:52.620222+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:07:52.620222+0000","last_undegraded":"2024-09-18T01:07:52.620222+0000","last_fullsized":"2024-09-18T01:07:52.620222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'43","reported_seq":715,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613876+0000","last_change":"2024-09-18T01:07:50.599672+0000","last_active":"2024-09-18T01:07:52.613876+0000","last_peered":"2024-09-18T01:07:52.613876+0000","last_clean":"2024-09-18T01:07:52.613876+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:07:52.613876+0000","last_undegraded":"2024-09-18T01:07:52.613876+0000","last_fullsized":"2024-09-18T01:07:52.613876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:33:16.474272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040002599999999999,"stat_sum":{"num_bytes":22988662,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":582,"num_read_kb":10604,"num_write":41,"num_write_kb":8952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.3","version":"43'74","reported_seq":582,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593517+0000","last_change":"2024-09-18T01:07:50.593768+0000","last_active":"2024-09-18T01:07:50.593517+0000","last_peered":"2024-09-18T01:07:50.593517+0000","last_clean":"2024-09-18T01:07:50.593517+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:07:50.593517+0000","last_undegraded":"2024-09-18T01:07:50.593517+0000","last_fullsized":"2024-09-18T01:07:50.593517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:23.958763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622472,"stat_sum":{"num_bytes":5914624,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":859,"num_read_kb":7864,"num_write":74,"num_write_kb":3507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613538+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:07:52.613538+0000","last_peered":"2024-09-18T01:07:52.613538+0000","last_clean":"2024-09-18T01:07:52.613538+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:07:52.613538+0000","last_undegraded":"2024-09-18T01:07:52.613538+0000","last_fullsized":"2024-09-18T01:07:52.613538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"43'81","reported_seq":740,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.652247+0000","last_change":"2024-09-18T01:07:50.673042+0000","last_active":"2024-09-18T01:07:52.652247+0000","last_peered":"2024-09-18T01:07:52.652247+0000","last_clean":"2024-09-18T01:07:52.652247+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:07:52.652247+0000","last_undegraded":"2024-09-18T01:07:52.652247+0000","last_fullsized":"2024-09-18T01:07:52.652247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:17.778103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366568,"stat_sum":{"num_bytes":15777539,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":811,"num_read_kb":20355,"num_write":137,"num_write_kb":8749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611667+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:07:52.611667+0000","last_peered":"2024-09-18T01:07:52.611667+0000","last_clean":"2024-09-18T01:07:52.611667+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:07:52.611667+0000","last_undegraded":"2024-09-18T01:07:52.611667+0000","last_fullsized":"2024-09-18T01:07:52.611667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'110","reported_seq":934,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.669676+0000","last_change":"2024-09-18T01:07:50.594666+0000","last_active":"2024-09-18T01:07:52.669676+0000","last_peered":"2024-09-18T01:07:52.669676+0000","last_clean":"2024-09-18T01:07:52.669676+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:07:52.669676+0000","last_undegraded":"2024-09-18T01:07:52.669676+0000","last_fullsized":"2024-09-18T01:07:52.669676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:26.969625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079864,"stat_sum":{"num_bytes":25597834,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":973,"num_read_kb":8239,"num_write":104,"num_write_kb":13556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611623+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:07:52.611623+0000","last_peered":"2024-09-18T01:07:52.611623+0000","last_clean":"2024-09-18T01:07:52.611623+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:07:52.611623+0000","last_undegraded":"2024-09-18T01:07:52.611623+0000","last_fullsized":"2024-09-18T01:07:52.611623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"48'79","reported_seq":676,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.594047+0000","last_change":"2024-09-18T01:07:53.641045+0000","last_active":"2024-09-18T01:07:54.594047+0000","last_peered":"2024-09-18T01:07:54.594047+0000","last_clean":"2024-09-18T01:07:54.594047+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:07:54.594047+0000","last_undegraded":"2024-09-18T01:07:54.594047+0000","last_fullsized":"2024-09-18T01:07:54.594047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:16.106450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073822199999999997,"stat_sum":{"num_bytes":16991697,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":518,"num_read_kb":4915,"num_write":74,"num_write_kb":12399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.613530+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:07:52.613530+0000","last_peered":"2024-09-18T01:07:52.613530+0000","last_clean":"2024-09-18T01:07:52.613530+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:07:52.613530+0000","last_undegraded":"2024-09-18T01:07:52.613530+0000","last_fullsized":"2024-09-18T01:07:52.613530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.611634+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:07:52.611634+0000","last_peered":"2024-09-18T01:07:52.611634+0000","last_clean":"2024-09-18T01:07:52.611634+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:07:52.611634+0000","last_undegraded":"2024-09-18T01:07:52.611634+0000","last_fullsized":"2024-09-18T01:07:52.611634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"47'34","reported_seq":307,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555197+0000","last_change":"2024-09-18T01:07:53.641041+0000","last_active":"2024-09-18T01:07:54.555197+0000","last_peered":"2024-09-18T01:07:54.555197+0000","last_clean":"2024-09-18T01:07:54.555197+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:07:54.555197+0000","last_undegraded":"2024-09-18T01:07:54.555197+0000","last_fullsized":"2024-09-18T01:07:54.555197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:20.348592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679694,"stat_sum":{"num_bytes":3981328,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":5099,"num_write":27,"num_write_kb":1804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.9","version":"38'30","reported_seq":320,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-18T01:07:50.593461+0000","last_change":"2024-09-18T01:07:50.593461+0000","last_active":"2024-09-18T01:07:50.593461+0000","last_peered":"2024-09-18T01:07:50.593461+0000","last_clean":"2024-09-18T01:07:50.593461+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:07:50.593461+0000","last_undegraded":"2024-09-18T01:07:50.593461+0000","last_fullsized":"2024-09-18T01:07:50.593461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:17:31.135951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034929399999999999,"stat_sum":{"num_bytes":7527681,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":193,"num_read_kb":12121,"num_write":23,"num_write_kb":5474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.a","version":"46'31","reported_seq":297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.624474+0000","last_change":"2024-09-18T01:07:50.673650+0000","last_active":"2024-09-18T01:07:52.624474+0000","last_peered":"2024-09-18T01:07:52.624474+0000","last_clean":"2024-09-18T01:07:52.624474+0000","last_became_active":"2024-09-18T01:07:28.974981+0000","last_became_peered":"2024-09-18T01:07:28.974981+0000","last_unstale":"2024-09-18T01:07:52.624474+0000","last_undegraded":"2024-09-18T01:07:52.624474+0000","last_fullsized":"2024-09-18T01:07:52.624474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:03.020699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046493499999999998,"stat_sum":{"num_bytes":4247889,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":165,"num_read_kb":5376,"num_write":26,"num_write_kb":2894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.b","version":"47'62","reported_seq":580,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T01:07:54.555469+0000","last_change":"2024-09-18T01:07:53.641181+0000","last_active":"2024-09-18T01:07:54.555469+0000","last_peered":"2024-09-18T01:07:54.555469+0000","last_clean":"2024-09-18T01:07:54.555469+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:07:54.555469+0000","last_undegraded":"2024-09-18T01:07:54.555469+0000","last_fullsized":"2024-09-18T01:07:54.555469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:20.405849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277499999999999,"stat_sum":{"num_bytes":10812299,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":509,"num_read_kb":14498,"num_write":78,"num_write_kb":5730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"1"}]},{"pgid":"3.c","version":"46'409","reported_seq":749,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.634590+0000","last_change":"2024-09-18T01:07:50.600021+0000","last_active":"2024-09-18T01:07:52.634590+0000","last_peered":"2024-09-18T01:07:52.634590+0000","last_clean":"2024-09-18T01:07:52.634590+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:07:52.634590+0000","last_undegraded":"2024-09-18T01:07:52.634590+0000","last_fullsized":"2024-09-18T01:07:52.634590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:08:38.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072939299999999995,"stat_sum":{"num_bytes":4696632,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2163,"num_read_kb":5694,"num_write":784,"num_write_kb":1716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.d","version":"46'61","reported_seq":436,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.721887+0000","last_change":"2024-09-18T01:07:50.674706+0000","last_active":"2024-09-18T01:07:52.721887+0000","last_peered":"2024-09-18T01:07:52.721887+0000","last_clean":"2024-09-18T01:07:52.721887+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:07:52.721887+0000","last_undegraded":"2024-09-18T01:07:52.721887+0000","last_fullsized":"2024-09-18T01:07:52.721887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:38:39.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072260800000000002,"stat_sum":{"num_bytes":8752607,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":346,"num_read_kb":9466,"num_write":51,"num_write_kb":10001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.e","version":"46'128","reported_seq":491,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.673879+0000","last_change":"2024-09-18T01:07:50.674842+0000","last_active":"2024-09-18T01:07:52.673879+0000","last_peered":"2024-09-18T01:07:52.673879+0000","last_clean":"2024-09-18T01:07:52.673879+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:07:52.673879+0000","last_undegraded":"2024-09-18T01:07:52.673879+0000","last_fullsized":"2024-09-18T01:07:52.673879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:48.391131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098800000000002,"stat_sum":{"num_bytes":10032599,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":772,"num_read_kb":10945,"num_write":223,"num_write_kb":6093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.f","version":"46'29","reported_seq":265,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.676003+0000","last_change":"2024-09-18T01:07:50.674619+0000","last_active":"2024-09-18T01:07:52.676003+0000","last_peered":"2024-09-18T01:07:52.676003+0000","last_clean":"2024-09-18T01:07:52.676003+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:07:52.676003+0000","last_undegraded":"2024-09-18T01:07:52.676003+0000","last_fullsized":"2024-09-18T01:07:52.676003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:15.729758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066955400000000005,"stat_sum":{"num_bytes":5769900,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":133,"num_read_kb":5056,"num_write":16,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.10","version":"46'62","reported_seq":407,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.661758+0000","last_change":"2024-09-18T01:07:50.673690+0000","last_active":"2024-09-18T01:07:52.661758+0000","last_peered":"2024-09-18T01:07:52.661758+0000","last_clean":"2024-09-18T01:07:52.661758+0000","last_became_active":"2024-09-18T01:07:28.976873+0000","last_became_peered":"2024-09-18T01:07:28.976873+0000","last_unstale":"2024-09-18T01:07:52.661758+0000","last_undegraded":"2024-09-18T01:07:52.661758+0000","last_fullsized":"2024-09-18T01:07:52.661758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:29.577702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045779700000000002,"stat_sum":{"num_bytes":8438077,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":310,"num_read_kb":6314,"num_write":59,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.11","version":"46'63","reported_seq":177,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.722347+0000","last_change":"2024-09-18T01:07:50.674145+0000","last_active":"2024-09-18T01:07:52.722347+0000","last_peered":"2024-09-18T01:07:52.722347+0000","last_clean":"2024-09-18T01:07:52.722347+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:07:52.722347+0000","last_undegraded":"2024-09-18T01:07:52.722347+0000","last_fullsized":"2024-09-18T01:07:52.722347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:26:41.996780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044894299999999999,"stat_sum":{"num_bytes":262144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":203,"num_read_kb":922,"num_write":91,"num_write_kb":559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.12","version":"46'79","reported_seq":317,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.630648+0000","last_change":"2024-09-18T01:07:50.594433+0000","last_active":"2024-09-18T01:07:52.630648+0000","last_peered":"2024-09-18T01:07:52.630648+0000","last_clean":"2024-09-18T01:07:52.630648+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:07:52.630648+0000","last_undegraded":"2024-09-18T01:07:52.630648+0000","last_fullsized":"2024-09-18T01:07:52.630648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:08.614064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093869099999999996,"stat_sum":{"num_bytes":3123076,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":233,"num_read_kb":3429,"num_write":63,"num_write_kb":2037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.13","version":"44'38","reported_seq":195,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:07:52.632614+0000","last_change":"2024-09-18T01:07:50.674463+0000","last_active":"2024-09-18T01:07:52.632614+0000","last_peered":"2024-09-18T01:07:52.632614+0000","last_clean":"2024-09-18T01:07:52.632614+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:07:52.632614+0000","last_undegraded":"2024-09-18T01:07:52.632614+0000","last_fullsized":"2024-09-18T01:07:52.632614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:55.753549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848800000000001,"stat_sum":{"num_bytes":1055082,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":54,"num_read_kb":2094,"num_write":21,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":328177377,"num_objects":315,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":315,"num_whiteouts":0,"num_read":18640,"num_read_kb":294008,"num_write":3307,"num_write_kb":189792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":248483840,"data_stored":247611507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2602,"ondisk_log_size":2602,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64148,"kb_used_data":32964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307692,"statfs":{"total":96636764160,"available":96571076608,"internally_reserved":0,"allocated":33755136,"data_stored":33396612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":9296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":9519104,"data_stored":9202353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62436,"kb_used_data":31248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309404,"statfs":{"total":96636764160,"available":96572829696,"internally_reserved":0,"allocated":31997952,"data_stored":31583815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47988,"kb_used_data":20960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323852,"statfs":{"total":96636764160,"available":96587624448,"internally_reserved":0,"allocated":21463040,"data_stored":21108591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59544,"kb_used_data":28356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312296,"statfs":{"total":96636764160,"available":96575791104,"internally_reserved":0,"allocated":29036544,"data_stored":28619512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65672,"kb_used_data":34484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306168,"statfs":{"total":96636764160,"available":96569516032,"internally_reserved":0,"allocated":35311616,"data_stored":34890339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74516,"kb_used_data":43324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297324,"statfs":{"total":96636764160,"available":96560459776,"internally_reserved":0,"allocated":44363776,"data_stored":43966716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69312,"kb_used_data":38116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302528,"statfs":{"total":96636764160,"available":96565788672,"internally_reserved":0,"allocated":39030784,"data_stored":38537310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37265408,"data_stored":37123191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29220864,"data_stored":29089429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21516288,"data_stored":21446726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":38490112,"data_stored":38299708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43753974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35512320,"data_stored":35383760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9965568,"data_stored":9933704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32581015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:07:56.404 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:07:56.405 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:07:56.405 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T01:07:56.405 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-18T01:07:56.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-09-18T01:07:56.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:07:56.466 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:56.465+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:56.550 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.551 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.559 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:07:56.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:07:56.567 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:56.566+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:56.606 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:07:56.690 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:56.689+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:56.721 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:56.721 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:56.732 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:07:56.769 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:56.768+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:56.777 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:56.777 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:56.796 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:56.813 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:56.832 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:56.851 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:56.852 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-18T01:07:56.869 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:56.872 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:56.868+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:56.886 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:56.886 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:56.887 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:56.887 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:56.887 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:56.887 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:56.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:07:56.910 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:56.910 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:56.970 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:56.969+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:57.021 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.021 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:57.021 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:57.021 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:57.021 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.030 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:07:57.072 INFO:teuthology.orchestra.run.smithi120.stdout:661 read 0xd1b449e thru 0xd1c3449 (0xefac bytes) 2024-09-18T01:07:57.072 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:57.071+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:57.072 INFO:teuthology.orchestra.run.smithi120.stdout:663 write 0x18f368d thru 0x18f4ee0 (0x1854 bytes) 2024-09-18T01:07:57.081 INFO:teuthology.orchestra.run.smithi120.stdout:664 writesame 0x1fc8fd7 thru 0x1fcde5c data_size 0x26(0x4e86 bytes) 2024-09-18T01:07:57.081 INFO:teuthology.orchestra.run.smithi120.stdout:665 read 0x8c5b51c thru 0x8c5e6a6 (0x318b bytes) 2024-09-18T01:07:57.081 INFO:teuthology.orchestra.run.smithi120.stdout:667 punch from 0xc6143e2 to 0xc6180a7, (0x3cc5 bytes) 2024-09-18T01:07:57.081 INFO:teuthology.orchestra.run.smithi120.stdout:668 read 0xd8b0289 thru 0xd8beba5 (0xe91d bytes) 2024-09-18T01:07:57.081 INFO:teuthology.orchestra.run.smithi120.stdout:670 write 0xc01eb76 thru 0xc02939a (0xa825 bytes) 2024-09-18T01:07:57.090 INFO:teuthology.orchestra.run.smithi120.stdout:671 write 0xd9b868b thru 0xd9c5e5b (0xd7d1 bytes) 2024-09-18T01:07:57.106 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.106 INFO:teuthology.orchestra.run.smithi120.stdout:673 punch from 0xde0fff1 to 0xde1f07b, (0xf08a bytes) 2024-09-18T01:07:57.107 INFO:teuthology.orchestra.run.smithi120.stdout:674 write 0x91a889a thru 0x91aab29 (0x2290 bytes) 2024-09-18T01:07:57.110 INFO:teuthology.orchestra.run.smithi120.stdout:675 writesame 0x7ab6c90 thru 0x7ab6dbf data_size 0x2(0x130 bytes) 2024-09-18T01:07:57.111 INFO:teuthology.orchestra.run.smithi120.stdout:676 write 0x8c28b1c thru 0x8c2bf56 (0x343b bytes) 2024-09-18T01:07:57.120 INFO:teuthology.orchestra.run.smithi120.stdout:677 read 0x78508cc thru 0x78583c5 (0x7afa bytes) 2024-09-18T01:07:57.121 INFO:teuthology.orchestra.run.smithi120.stdout:679 write 0xcb8222 thru 0xcc5efd (0xdcdc bytes) 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961353755000001, 2024-09-18T01:07:57.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.143 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:07:57.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:07:57.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.148 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961334951000001, 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.150 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:07:57.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961318382999998, 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:07:57.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961302196999998, 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:07:57.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961290628, 2024-09-18T01:07:57.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.170 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.175 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961281262, 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:07:57.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:07:57.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:07:57.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:07:57.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961269079000001, 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:07:57.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961253383999999, 2024-09-18T01:07:57.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961236230000001, 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:07:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961214064, 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:07:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:07:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961197614, 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961188159999999, 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:07:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:07:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:07:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961154108999999, 2024-09-18T01:07:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:07:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:07:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961134385000001, 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961116147999999, 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:07:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961095573000001, 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961076345999999, 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:07:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:07:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:07:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961056993, 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.961038338000002, 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:07:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:07:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:07:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:07:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.952256388999999, 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:07:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:07:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:57.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.265 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.267 INFO:teuthology.orchestra.run.smithi120.stdout:680 trunc from 0xe9956b4 to 0xb6166a 2024-09-18T01:07:57.267 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:57.266+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:57.268 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:07:57.270 INFO:teuthology.orchestra.run.smithi120.stdout:682 punch from 0x476bb0 to 0x478137, (0x1587 bytes) 2024-09-18T01:07:57.270 INFO:teuthology.orchestra.run.smithi120.stdout:683 read 0x41d71e thru 0x4240ab (0x698e bytes) 2024-09-18T01:07:57.270 INFO:teuthology.orchestra.run.smithi120.stdout:684 punch from 0x2530db to 0x25fd58, (0xcc7d bytes) 2024-09-18T01:07:57.270 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:688 read 0x2e0f3a thru 0x2e0f46 (0xd bytes) 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:690 trunc from 0xb6166a to 0xabd99a7 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:691 write 0x620ca4f thru 0x620f2cd (0x287f bytes) 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:692 read 0x69110cb thru 0x691e3ac (0xd2e2 bytes) 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:693 read 0x6549f89 thru 0x65556e2 (0xb75a bytes) 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:695 write 0x3eda086 thru 0x3ee6307 (0xc282 bytes) 2024-09-18T01:07:57.271 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.272 INFO:teuthology.orchestra.run.smithi120.stdout:698 write 0xe74a028 thru 0xe751995 (0x796e bytes) 2024-09-18T01:07:57.279 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.279 INFO:teuthology.orchestra.run.smithi120.stdout:702 write 0x2fa2e thru 0x38a82 (0x9055 bytes) 2024-09-18T01:07:57.286 INFO:teuthology.orchestra.run.smithi120.stdout:703 read 0xbe449c thru 0xbed048 (0x8bad bytes) 2024-09-18T01:07:57.286 INFO:teuthology.orchestra.run.smithi120.stdout:704 punch from 0x7d3a876 to 0x7d427b3, (0x7f3d bytes) 2024-09-18T01:07:57.286 INFO:teuthology.orchestra.run.smithi120.stdout:705 punch from 0xb19675e to 0xb197735, (0xfd7 bytes) 2024-09-18T01:07:57.287 INFO:teuthology.orchestra.run.smithi120.stdout:706 read 0x16ec4fb thru 0x16f9f94 (0xda9a bytes) 2024-09-18T01:07:57.287 INFO:teuthology.orchestra.run.smithi120.stdout:707 write 0xd303f43 thru 0xd30650d (0x25cb bytes) 2024-09-18T01:07:57.291 INFO:teuthology.orchestra.run.smithi120.stdout:708 punch from 0x1fbb39a to 0x1fc3b27, (0x878d bytes) 2024-09-18T01:07:57.308 INFO:teuthology.orchestra.run.smithi120.stdout:710 trunc from 0xe751996 to 0xf0e94e 2024-09-18T01:07:57.327 INFO:teuthology.orchestra.run.smithi120.stdout:711 read 0x760a28 thru 0x76d89f (0xce78 bytes) 2024-09-18T01:07:57.344 INFO:teuthology.orchestra.run.smithi120.stdout:712 writesame 0xa73f030 thru 0xa740766 data_size 0x7(0x1737 bytes) 2024-09-18T01:07:57.354 INFO:teuthology.orchestra.run.smithi120.stdout:714 trunc from 0xa740767 to 0x293e0f0 2024-09-18T01:07:57.368 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:57.367+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:57.378 INFO:teuthology.orchestra.run.smithi120.stdout:715 write 0xe434e09 thru 0xe43fba9 (0xada1 bytes) 2024-09-18T01:07:57.389 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.390 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:57.390 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:57.390 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.393 INFO:teuthology.orchestra.run.smithi120.stdout:716 read 0x219b524 thru 0x219c54d (0x102a bytes) 2024-09-18T01:07:57.393 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.393 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.394 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.394 INFO:teuthology.orchestra.run.smithi120.stdout:726 punch from 0xa7e061a to 0xa7efd80, (0xf766 bytes) 2024-09-18T01:07:57.394 INFO:teuthology.orchestra.run.smithi120.stdout:727 writesame 0x9cb1d80 thru 0x9cb808f data_size 0x28(0x6310 bytes) 2024-09-18T01:07:57.399 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:07:57.403 INFO:teuthology.orchestra.run.smithi120.stdout:729 write 0xbf17d7d thru 0xbf1e134 (0x63b8 bytes) 2024-09-18T01:07:57.415 INFO:teuthology.orchestra.run.smithi120.stdout:730 punch from 0x48212fc to 0x482c29a, (0xaf9e bytes) 2024-09-18T01:07:57.415 INFO:teuthology.orchestra.run.smithi120.stdout:734 read 0x46d901e thru 0x46e1fe6 (0x8fc9 bytes) 2024-09-18T01:07:57.415 INFO:teuthology.orchestra.run.smithi120.stdout:736 write 0x71f4daa thru 0x7200f1f (0xc176 bytes) 2024-09-18T01:07:57.423 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.423 INFO:teuthology.orchestra.run.smithi120.stdout:738 writesame 0x1a0eb85 thru 0x1a18a0f data_size 0x3(0x9e8b bytes) 2024-09-18T01:07:57.432 INFO:teuthology.orchestra.run.smithi120.stdout:739 writesame 0x5287425 thru 0x528ca0e data_size 0x7(0x55ea bytes) 2024-09-18T01:07:57.439 INFO:teuthology.orchestra.run.smithi120.stdout:740 read 0xd4ffbf5 thru 0xd507a7f (0x7e8b bytes) 2024-09-18T01:07:57.439 INFO:teuthology.orchestra.run.smithi120.stdout:741 write 0xa04c32b thru 0xa056823 (0xa4f9 bytes) 2024-09-18T01:07:57.447 INFO:teuthology.orchestra.run.smithi120.stdout:743 writesame 0x173bd3e thru 0x1745edd data_size 0x20(0xa1a0 bytes) 2024-09-18T01:07:57.455 INFO:teuthology.orchestra.run.smithi120.stdout:744 write 0x9ccb7b8 thru 0x9cd997a (0xe1c3 bytes) 2024-09-18T01:07:57.464 INFO:teuthology.orchestra.run.smithi120.stdout:745 writesame 0xb4e209b thru 0xb4e76f0 data_size 0x2b(0x5656 bytes) 2024-09-18T01:07:57.469 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:57.468+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:57.470 INFO:teuthology.orchestra.run.smithi120.stdout:746 write 0x98aa6d8 thru 0x98b6f86 (0xc8af bytes) 2024-09-18T01:07:57.485 INFO:teuthology.orchestra.run.smithi120.stdout:747 write 0xdf045b2 thru 0xdf0f7b0 (0xb1ff bytes) 2024-09-18T01:07:57.490 INFO:teuthology.orchestra.run.smithi120.stdout:748 write 0xd8995b2 thru 0xd89b646 (0x2095 bytes) 2024-09-18T01:07:57.503 INFO:teuthology.orchestra.run.smithi120.stdout:749 trunc from 0xe43fbaa to 0x86677dc 2024-09-18T01:07:57.518 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.518 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:57.518 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:57.518 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:57.518 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.527 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:07:57.534 INFO:teuthology.orchestra.run.smithi120.stdout:750 read 0x4f8fd4 thru 0x504b53 (0xbb80 bytes) 2024-09-18T01:07:57.537 INFO:teuthology.orchestra.run.smithi120.stdout:751 trunc from 0x86677dc to 0x6891aec 2024-09-18T01:07:57.557 INFO:teuthology.orchestra.run.smithi120.stdout:752 write 0xa6b8396 thru 0xa6c1d95 (0x9a00 bytes) 2024-09-18T01:07:57.567 INFO:teuthology.orchestra.run.smithi120.stdout:754 punch from 0x8e3db5d to 0x8e3fac3, (0x1f66 bytes) 2024-09-18T01:07:57.567 INFO:teuthology.orchestra.run.smithi120.stdout:756 writesame 0x1761fcf thru 0x1767033 data_size 0x1(0x5065 bytes) 2024-09-18T01:07:57.570 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:57.569+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:57.581 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.581 INFO:teuthology.orchestra.run.smithi120.stdout:760 read 0x61df64 thru 0x620cea (0x2d87 bytes) 2024-09-18T01:07:57.582 INFO:teuthology.orchestra.run.smithi120.stdout:762 write 0x9df9aff thru 0x9e06d90 (0xd292 bytes) 2024-09-18T01:07:57.594 INFO:teuthology.orchestra.run.smithi120.stdout:763 trunc from 0xa6c1d96 to 0x564f927 2024-09-18T01:07:57.614 INFO:teuthology.orchestra.run.smithi120.stdout:764 trunc from 0x564f927 to 0xa30487f 2024-09-18T01:07:57.620 INFO:teuthology.orchestra.run.smithi120.stdout:766 punch from 0x838e7f1 to 0x839a303, (0xbb12 bytes) 2024-09-18T01:07:57.620 INFO:teuthology.orchestra.run.smithi120.stdout:767 writesame 0x864e98b thru 0x8653c0c data_size 0x2(0x5282 bytes) 2024-09-18T01:07:57.627 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:07:57.633 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:57.633 INFO:teuthology.orchestra.run.smithi120.stdout:770 read 0x2646a0d thru 0x264971f (0x2d13 bytes) 2024-09-18T01:07:57.635 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-18T01:07:57.645 INFO:teuthology.orchestra.run.smithi120.stdout:771 write 0xa71ed94 thru 0xa7283c9 (0x9636 bytes) 2024-09-18T01:07:57.657 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.716330634000002, 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:07:57.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:07:57.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:783 3.0 e28/23 3:04cd6778:::rbd_data.11769efc726f.0000000000000005:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.610139508, 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026120100000000001, 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:07:57.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045917+0000", 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0500000000000003e-07 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046106+0000", 2024-09-18T01:07:57.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018893600000000001 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046125+0000", 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9128e-05 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046137+0000", 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046199+0000", 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2520000000000004e-05 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046222+0000", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2554000000000001e-05 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048480+0000", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002258047 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048528+0000", 2024-09-18T01:07:57.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8241000000000001e-05 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.588402594000001, 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.670 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:07:57.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.538028533999999, 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:07:57.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1568 3.9 e28/23 3:92680351:::rbd_data.11769efc726f.000000000000002b:head v 28'2, mlcod=0'0)", 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:57.679 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.529454155, 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026420340000000001, 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126603+0000", 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.215e-06 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126978+0000", 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037491299999999999 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126997+0000", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9375e-05 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127007+0000", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127067+0000", 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9812000000000003e-05 2024-09-18T01:07:57.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127090+0000", 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3828e-05 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129205+0000", 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002114215 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129243+0000", 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8751000000000002e-05 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.684 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2196 3.4 e28/23 3:26396459:::rbd_data.11769efc726f.000000000000003e:head v 28'7, mlcod=26'5)", 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.449709768999998, 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028009300000000001, 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206347+0000", 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.341e-06 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206440+0000", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3080999999999999e-05 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206458+0000", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7555999999999999e-05 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206467+0000", 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1519999999999998e-06 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206543+0000", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5899e-05 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206565+0000", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2497000000000002e-05 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209098+0000", 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025328080000000001 2024-09-18T01:07:57.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209147+0000", 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8596e-05 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.448135317999999, 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:07:57.689 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:07:57.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:07:57.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:07:57.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.446539569999999, 2024-09-18T01:07:57.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:07:57.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:07:57.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.445728951, 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:07:57.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.395181458, 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:07:57.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:3134 3.4 e29/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 29'16, mlcod=28'13)", 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.447807523000002, 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031975599999999999, 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:57.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208255+0000", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.319e-06 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208262+0000", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4309999999999997e-06 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208267+0000", 2024-09-18T01:07:57.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0370000000000002e-06 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208575+0000", 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030814599999999997 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208594+0000", 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8635999999999999e-05 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211406+0000", 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028115670000000001 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211446+0000", 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0021999999999997e-05 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.06235614, 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:07:57.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:07:57.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.498756083, 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:07:57.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:07:57.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 162 entries 38'77 (0'0) error 3:0796c97e:::rbd_data.117611b60058.0000000000000155:head by client.4470.0:13249 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'75)", 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.482022096, 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030461970000000001, 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174034+0000", 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174041+0000", 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174060+0000", 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8848999999999999e-05 2024-09-18T01:07:57.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174069+0000", 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8950000000000004e-06 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174139+0000", 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0235999999999996e-05 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177080+0000", 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029401760000000001 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.480994997, 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:07:57.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.478338294, 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.733 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.737 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:15261 3.0 e43/23 3:03957b90:::rbd_data.1176492bc9ca.000000000000000c:head v 43'81, mlcod=40'79)", 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.0656390719999997, 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031977450000000001, 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590418+0000", 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4890000000000001e-06 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590594+0000", 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017577099999999999 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590610+0000", 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6327000000000001e-05 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590618+0000", 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3839999999999997e-06 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590682+0000", 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3596000000000006e-05 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590708+0000", 2024-09-18T01:07:57.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6089e-05 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593568+0000", 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028597789999999998 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593614+0000", 2024-09-18T01:07:57.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-05 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8956907510000001, 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:57.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:07:57.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.749 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4813772919999999, 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.750 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:07:57.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:07:57.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:07:57.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:27934 3.0 e50/23 3:06a5839e:::rbd_data.1176e641238f.0000000000000080:head v 50'87, mlcod=49'85)", 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:07:57.757 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.46039983299999998, 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030122230000000001, 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:57.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5199999999999996e-07 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:57.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195662+0000", 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8889999999999999e-06 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195673+0000", 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0913999999999999e-05 2024-09-18T01:07:57.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195678+0000", 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1869999999999996e-06 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195733+0000", 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4197999999999998e-05 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195755+0000", 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2790999999999999e-05 2024-09-18T01:07:57.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.198622+0000", 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002866674 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:57.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.198668+0000", 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5818000000000001e-05 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:57.765 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.767 INFO:teuthology.orchestra.run.smithi120.stdout:772 write 0xd103060 thru 0xd10b4d4 (0x8475 bytes) 2024-09-18T01:07:57.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:07:57.769 INFO:teuthology.orchestra.run.smithi120.stdout:774 punch from 0x5919868 to 0x59295ac, (0xfd44 bytes) 2024-09-18T01:07:57.769 INFO:teuthology.orchestra.run.smithi120.stdout:776 writesame 0xc8bbd8e thru 0xc8c4e24 data_size 0xb(0x9097 bytes) 2024-09-18T01:07:57.770 INFO:teuthology.orchestra.run.smithi120.stdout:777 writesame 0x20d0005 thru 0x20d512a data_size 0x2f(0x5126 bytes) 2024-09-18T01:07:57.770 INFO:teuthology.orchestra.run.smithi120.stdout:778 write 0x94f7d54 thru 0x94fc887 (0x4b34 bytes) 2024-09-18T01:07:57.770 INFO:teuthology.orchestra.run.smithi120.stdout:779 trunc from 0xd10b4d5 to 0xbec4dc1 2024-09-18T01:07:57.770 INFO:teuthology.orchestra.run.smithi120.stdout:780 write 0x302e77 thru 0x30ae37 (0x7fc1 bytes) 2024-09-18T01:07:57.770 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:57.766+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:57.868 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:57.867+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:57.911 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:57.912 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:57.912 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:07:57.912 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:57.920 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:07:57.969 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:57.968+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:58.023 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:58.023 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:07:58.023 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:07:58.023 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:07:58.023 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:58.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:07:58.073 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:58.072+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:58.170 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:58.169+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:58.178 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:07:58.178 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:07:58.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:07:58.178 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8294 3.7 3:e3e34f1e:::rbd_data.11762d3aac1.0000000000000000:head [write 0~524288 [fadvise_sequential+fadvise_nocache],zero 524288~524288,write 1048576~524288 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~1048576 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.590753439, 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027761263000000001, 2024-09-18T01:07:58.179 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.180 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8294 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:58.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586908+0000", 2024-09-18T01:07:58.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022325259999999999 2024-09-18T01:07:58.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586922+0000", 2024-09-18T01:07:58.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4061e-05 2024-09-18T01:07:58.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586931+0000", 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2089999999999994e-06 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587661+0000", 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00073032299999999995 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587811+0000", 2024-09-18T01:07:58.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000150047 2024-09-18T01:07:58.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.588205+0000", 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039331099999999998 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610502+0000", 2024-09-18T01:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022297437 2024-09-18T01:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612252+0000", 2024-09-18T01:07:58.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017498940000000001 2024-09-18T01:07:58.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612391+0000", 2024-09-18T01:07:58.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013893099999999999 2024-09-18T01:07:58.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612436+0000", 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5093999999999999e-05 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8316 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 1048576~163840 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.588381811000001, 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012174538, 2024-09-18T01:07:58.194 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.195 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8316 2024-09-18T01:07:58.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.196 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:07:58.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587047+0000", 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0400000000000002e-07 2024-09-18T01:07:58.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587049+0000", 2024-09-18T01:07:58.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.728e-06 2024-09-18T01:07:58.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587055+0000", 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9909999999999999e-06 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587060+0000", 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5360000000000003e-06 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595805+0000", 2024-09-18T01:07:58.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087449510000000008 2024-09-18T01:07:58.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595925+0000", 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012037200000000001 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599221+0000", 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032955559999999998 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8324 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 1572864~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.588075856, 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019081754999999999, 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8324 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.206 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:58.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587353+0000", 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2399999999999998e-07 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587356+0000", 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9450000000000002e-06 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587363+0000", 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.452e-06 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587369+0000", 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5659999999999997e-06 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599368+0000", 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011999604000000001 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599490+0000", 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121357 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606434+0000", 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006944207 2024-09-18T01:07:58.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8341 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.587999359000001, 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02321252, 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8341 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587435+0000", 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9610000000000002e-06 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587442+0000", 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7000000000000002e-06 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587447+0000", 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606471+0000", 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019024926000000001 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606525+0000", 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3958000000000001e-05 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610641+0000", 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041158519999999997 2024-09-18T01:07:58.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8348 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.581543200999999, 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013699756, 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8348 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593887+0000", 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2400000000000002e-06 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593902+0000", 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4421e-05 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593922+0000", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0020000000000001e-05 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593929+0000", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7759999999999997e-06 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594130+0000", 2024-09-18T01:07:58.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020075599999999999 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594241+0000", 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001111 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594643+0000", 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00040180700000000001 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600371+0000", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057278600000000004 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607482+0000", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071108730000000002 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607536+0000", 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4230000000000001e-05 2024-09-18T01:07:58.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607585+0000", 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8673e-05 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8350 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.581143783999998, 2024-09-18T01:07:58.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017945170999999999, 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8350 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:58.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594285+0000", 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000004e-07 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594290+0000", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1710000000000001e-06 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594301+0000", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0499000000000001e-05 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594309+0000", 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9650000000000008e-06 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610528+0000", 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016218713999999999 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610576+0000", 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8368999999999997e-05 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612230+0000", 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016536280000000001 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8390 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.579409019, 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01144247, 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8390 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596020+0000", 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596022+0000", 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8310000000000001e-06 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596029+0000", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6549999999999998e-06 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596044+0000", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4574e-05 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600514+0000", 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044701209999999996 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600615+0000", 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100936 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607462+0000", 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068468139999999997 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8399 3.2 3:44ca0a04:::rbd_data.11762d3aac1.0000000000000006:head [write 0~3670016 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache],zero 5767168~524288,zero 7602176~262144,zero 15990784~218866,write 16209650~22755 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.56171625, 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029285142, 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8399 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613714+0000", 2024-09-18T01:07:58.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9180000000000001e-06 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615140+0000", 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014266820000000001 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615157+0000", 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6330000000000001e-05 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615166+0000", 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615300+0000", 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013375400000000001 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615497+0000", 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019682799999999999 2024-09-18T01:07:58.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.616109+0000", 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061267699999999995 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642827+0000", 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026717366999999999 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642859+0000", 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2777999999999997e-05 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642961+0000", 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101137 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642997+0000", 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6449000000000003e-05 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8447 3.2 3:452cb511:::rbd_data.1176ec0d676b.0000000000000058:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.560044564999998, 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013050262, 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.240 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8447 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615386+0000", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615392+0000", 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.037e-06 2024-09-18T01:07:58.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615396+0000", 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628305+0000", 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012908974 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628434+0000", 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012840200000000001 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8457 3.2 3:46e71cef:::rbd_data.1176ec0d676b.000000000000006a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.560003084999998, 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013141890999999999, 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.244 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8457 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615426+0000", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615428+0000", 2024-09-18T01:07:58.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.046e-06 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615433+0000", 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615437+0000", 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9380000000000002e-06 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628468+0000", 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013030570999999999 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628567+0000", 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9035000000000003e-05 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:8397 3.5 e34/23 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head v 34'8, mlcod=31'6)", 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.556481482999999, 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030584585000000001, 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627937+0000", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089904949999999994 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627956+0000", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8984999999999999e-05 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627964+0000", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5129999999999999e-06 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628036+0000", 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2004999999999999e-05 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628066+0000", 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9569000000000001e-05 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649485+0000", 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021419779 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649531+0000", 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5948999999999997e-05 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8785 3.2 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head [write 0~1146641 [fadvise_sequential+fadvise_nocache],zero 1146641~426223,write 1572864~2621440 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.549980497, 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028310961999999999, 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8785 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625449+0000", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.773e-06 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627661+0000", 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022114980000000001 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627688+0000", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6988e-05 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627697+0000", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8710000000000003e-06 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629596+0000", 2024-09-18T01:07:58.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001899155 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630212+0000", 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061557099999999998 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630882+0000", 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00067058700000000005 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649558+0000", 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018675605000000001 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653628+0000", 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004070097 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653707+0000", 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9645000000000001e-05 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653759+0000", 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1171999999999997e-05 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8787 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.543261946000001, 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021599920000000002, 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8787 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632169+0000", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632181+0000", 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1564e-05 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632203+0000", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632227+0000", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3484000000000001e-05 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632310+0000", 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2891000000000003e-05 2024-09-18T01:07:58.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632426+0000", 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116288 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635044+0000", 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026176419999999999 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645349+0000", 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010305527 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653684+0000", 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083345120000000005 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653725+0000", 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1119999999999999e-05 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653766+0000", 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1287e-05 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8824 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.541669820999999, 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011546005, 2024-09-18T01:07:58.265 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8824 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633759+0000", 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633763+0000", 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1189999999999999e-06 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633770+0000", 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9129999999999997e-06 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633775+0000", 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645111+0000", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011336061999999999 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645226+0000", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114784 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645304+0000", 2024-09-18T01:07:58.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8510999999999994e-05 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8960 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.540710344000001, 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.014368275, 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.270 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8960 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1200000000000003e-07 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634720+0000", 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634725+0000", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634735+0000", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0176999999999999e-05 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643266+0000", 2024-09-18T01:07:58.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085303800000000006 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643358+0000", 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1699000000000004e-05 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649086+0000", 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057285180000000002 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8965 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 786432~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.53992886, 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015273905000000001, 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8965 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635500+0000", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-06 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635505+0000", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635516+0000", 2024-09-18T01:07:58.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1089e-05 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635529+0000", 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2754e-05 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649126+0000", 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013596964 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649214+0000", 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7905000000000006e-05 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.650773+0000", 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001559305 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8973 3.1e 3:7e62f05a:::rbd_data.1176ec0d676b.0000000000000012:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.539886840000001, 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01383679, 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8973 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635542+0000", 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9599999999999999e-07 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635544+0000", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4109999999999999e-06 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635551+0000", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8109999999999997e-06 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635556+0000", 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8550000000000001e-06 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646171+0000", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010615342 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646218+0000", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7259000000000003e-05 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649378+0000", 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031596160000000001 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8982 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1310720~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.539865469999999, 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015554682, 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8982 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635565+0000", 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8470000000000001e-06 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635571+0000", 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7219999999999996e-06 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635575+0000", 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649410+0000", 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013835055000000001 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649481+0000", 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0820000000000003e-05 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.651117+0000", 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016363770000000001 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9166 3.18 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.520283493000001, 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028895934000000002, 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9166 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655147+0000", 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9879999999999999e-06 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656773+0000", 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00162633 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656788+0000", 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4547e-05 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656795+0000", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2989999999999999e-06 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656857+0000", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2203000000000001e-05 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656994+0000", 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013699599999999999 2024-09-18T01:07:58.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 0", 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657438+0000", 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044429499999999999 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.682244+0000", 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024805437999999999 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683944+0000", 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001699926 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683993+0000", 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9447999999999998e-05 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.684041+0000", 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7463999999999998e-05 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9173 3.19 e34/23 3:9b92acfd:::rbd_data.11762d3aac1.000000000000000e:head v 34'19, mlcod=31'17)", 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.508131703, 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016520503999999998, 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667297+0000", 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3300000000000001e-07 2024-09-18T01:07:58.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668283+0000", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098571899999999992 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668305+0000", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1665999999999999e-05 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668317+0000", 2024-09-18T01:07:58.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2013e-05 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668402+0000", 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4970999999999997e-05 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668429+0000", 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7416e-05 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:07:58.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683769+0000", 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015339999999999999 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683817+0000", 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7886000000000002e-05 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:07:58.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:07:58.301 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:58.300+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:58.402 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:58.401+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:58.407 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3.47feccca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.684616373, 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.68463808800000003, 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:07:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-18T01:07:58.412 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:58.416 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:07:58.503 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:07:58.502+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:07:58.518 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:58.519 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:58.527 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:07:58.604 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:58.603+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:58.632 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.038855744999999, 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:07:58.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.030524072999999, 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:07:58.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:58.642 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.027183219000001, 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:07:58.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.999728390000001, 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.647 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:07:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.965763073000002, 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:07:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.884795487, 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:07:58.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.884773649, 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:07:58.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9944283489999997, 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:07:58.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17763 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 98304~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9929118419999998, 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092008709999999994, 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17763 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637405+0000", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637408+0000", 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.574e-06 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637416+0000", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2450000000000001e-06 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637422+0000", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3820000000000001e-06 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.644983+0000", 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0075605960000000002 2024-09-18T01:07:58.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.645035+0000", 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1925000000000002e-05 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646605+0000", 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015706019999999999 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9928854840000003, 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:07:58.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9928643270000004, 2024-09-18T01:07:58.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:07:58.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:07:58.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9928455129999998, 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:07:58.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:07:58.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5393387170000001, 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.690 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:07:58.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:07:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.539327337, 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:07:58.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:07:58.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.539313417, 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:07:58.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28135 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.2516053499999999, 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095049399999999999, 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28135 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6e-07 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378713+0000", 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.699e-06 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378716+0000", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.507e-06 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378719+0000", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6170000000000001e-06 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.381793+0000", 2024-09-18T01:07:58.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030739780000000002 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.381869+0000", 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6171e-05 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.382251+0000", 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038233000000000001 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.387988+0000", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057363559999999997 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388151+0000", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163695 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388189+0000", 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7999e-05 2024-09-18T01:07:58.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388216+0000", 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6327999999999999e-05 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28241 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.470593+0000", 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.159722699, 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0089980489999999993, 2024-09-18T01:07:58.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28241 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470593+0000", 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470593+0000", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470594+0000", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470595+0000", 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470601+0000", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470604+0000", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9519999999999999e-06 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.474305+0000", 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037016549999999999 2024-09-18T01:07:58.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.474359+0000", 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3177999999999998e-05 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.474571+0000", 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000212022 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479485+0000", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049146329999999998 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479528+0000", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2315999999999999e-05 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479557+0000", 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9170999999999999e-05 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479591+0000", 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4742e-05 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28242 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.470607+0000", 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.159709155, 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0090527119999999992, 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28242 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470607+0000", 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470607+0000", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470607+0000", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470609+0000", 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.579e-06 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470612+0000", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1370000000000002e-06 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.470615+0000", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-06 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.475021+0000", 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004406526 2024-09-18T01:07:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.475073+0000", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1649e-05 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.475296+0000", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000223506 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479519+0000", 2024-09-18T01:07:58.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042226140000000004 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479598+0000", 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8877999999999999e-05 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479626+0000", 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8138999999999999e-05 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.479660+0000", 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3822999999999998e-05 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.062778832, 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:07:58.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0627673529999999, 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:07:58.732 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:07:58.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:07:58.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:07:58.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:07:58.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:07:58.738 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:58.739 INFO:teuthology.orchestra.run.smithi120.stdout:782 writesame 0xbc9980e thru 0xbca13e8 data_size 0xd(0x7bdb bytes) 2024-09-18T01:07:58.740 INFO:teuthology.orchestra.run.smithi120.stdout:783 read 0x235b607 thru 0x2361b18 (0x6512 bytes) 2024-09-18T01:07:58.740 INFO:teuthology.orchestra.run.smithi120.stdout:784 writesame 0x1419aa9 thru 0x1422ab5 data_size 0x1(0x900d bytes) 2024-09-18T01:07:58.740 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:07:58.740 INFO:teuthology.orchestra.run.smithi120.stdout:786 clone 9 order 24 su 65536 sc 15 2024-09-18T01:07:58.741 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:07:58.741 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:58.738+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:58.841 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:07:58.840+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:07:58.885 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:58.886 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:58.886 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:07:58.886 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:58.894 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:07:58.927 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:07:58.978 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:58.977+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:59.037 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:07:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:07:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:07:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:07:59.037 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:07:59.042 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:59.041+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:59.046 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:07:59.046 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:07:59.047 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:07:59.063 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:07:59.063 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:07:59.079 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:07:59.079 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:07:59.094 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:07:59.094 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:07:59.110 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:07:59.111 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:07:59.129 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:07:59.129 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:07:59.143 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:07:59.143 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:07:59.144 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:59.142+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:59.160 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:07:59.160 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:07:59.189 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.190 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.198 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:07:59.244 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:59.243+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:07:59.342 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.342 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.344 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:07:59.343+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:07:59.351 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:07:59.446 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:07:59.445+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:07:59.494 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.494 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:07:59.546 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:07:59.545+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:07:59.611 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.611 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.620 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:07:59.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:07:59.647 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:59.646+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:59.748 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:07:59.747+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:07:59.766 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.766 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.775 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:07:59.849 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:07:59.848+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:07:59.919 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:07:59.920 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:07:59.928 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:07:59.950 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:07:59.949+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.041 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:00.086 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:00.085+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:00.144 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.145 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.152 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:00.151+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:00.157 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:00.253 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:00.252+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:00.300 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.300 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.309 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:00.354 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:00.354+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:00.451 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.451 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.455 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:00.454+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:00.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:00.538 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:08:00.556 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:00.554+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:00.585 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.585 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:00.689 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:00.685+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:00.757 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:00.756+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:00.810 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.810 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:00.859 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:00.858+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:00.960 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:00.959+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:00.964 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:00.965 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:00.973 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:01.061 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:01.060+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:01.075 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:01.075 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:01.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:01.161 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:01.160+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:01.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:01.186 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:01.186 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:01.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:01.262 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:01.261+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:01.292 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:01.293 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:01.310 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:01.310 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:01.327 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:01.327 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:01.343 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:01.344 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:01.344 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.402844756, 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:01.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:783 3.0 e28/23 3:04cd6778:::rbd_data.11769efc726f.0000000000000005:head v 28'2, mlcod=0'0)", 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.296653630000002, 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026120100000000001, 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045916+0000", 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.045917+0000", 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0500000000000003e-07 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046106+0000", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018893600000000001 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046125+0000", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9128e-05 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046137+0000", 2024-09-18T01:08:01.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046199+0000", 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2520000000000004e-05 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.046222+0000", 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2554000000000001e-05 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048480+0000", 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002258047 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.048528+0000", 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8241000000000001e-05 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:01.354 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.274916716, 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:01.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:01.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.224542656000001, 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.359 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:01.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:01.363 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1568 3.9 e28/23 3:92680351:::rbd_data.11769efc726f.000000000000002b:head v 28'2, mlcod=0'0)", 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.215968277000002, 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026420340000000001, 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126601+0000", 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126603+0000", 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.215e-06 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126978+0000", 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037491299999999999 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.126997+0000", 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9375e-05 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127007+0000", 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9250000000000007e-06 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127067+0000", 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9812000000000003e-05 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.127090+0000", 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3828e-05 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129205+0000", 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002114215 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.129243+0000", 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8751000000000002e-05 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2196 3.4 e28/23 3:26396459:::rbd_data.11769efc726f.000000000000003e:head v 28'7, mlcod=26'5)", 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.136223891, 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028009300000000001, 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206346+0000", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206347+0000", 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.341e-06 2024-09-18T01:08:01.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206440+0000", 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3080999999999999e-05 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206458+0000", 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7555999999999999e-05 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206467+0000", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1519999999999998e-06 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206543+0000", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5899e-05 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206565+0000", 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2497000000000002e-05 2024-09-18T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209098+0000", 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025328080000000001 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209147+0000", 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8596e-05 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.13464944, 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.374 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:01.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.133053692000001, 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:01.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.384 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.132243073000001, 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:01.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.389 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.081695580000002, 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:3134 3.4 e29/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 29'16, mlcod=28'13)", 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 25.134321645, 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031975599999999999, 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208248+0000", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208255+0000", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.319e-06 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208262+0000", 2024-09-18T01:08:01.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4309999999999997e-06 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208267+0000", 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0370000000000002e-06 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208575+0000", 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030814599999999997 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.208594+0000", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8635999999999999e-05 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211406+0000", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028115670000000001 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:36.211446+0000", 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0021999999999997e-05 2024-09-18T01:08:01.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.748870262000001, 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.400 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:01.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:01.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.185270204999998, 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:01.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:01.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 162 entries 38'77 (0'0) error 3:0796c97e:::rbd_data.117611b60058.0000000000000155:head by client.4470.0:13249 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'75)", 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.168536218, 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030461970000000001, 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.409 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174033+0000", 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174034+0000", 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174041+0000", 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174060+0000", 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8848999999999999e-05 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174069+0000", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8950000000000004e-06 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.174139+0000", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0235999999999996e-05 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177080+0000", 2024-09-18T01:08:01.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029401760000000001 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.167509119000002, 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.413 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.164852415999999, 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:01.417 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:01.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:15261 3.0 e43/23 3:03957b90:::rbd_data.1176492bc9ca.000000000000000c:head v 43'81, mlcod=40'79)", 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.752153194, 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031977450000000001, 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590416+0000", 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590418+0000", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4890000000000001e-06 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590594+0000", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017577099999999999 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590610+0000", 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6327000000000001e-05 2024-09-18T01:08:01.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590618+0000", 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3839999999999997e-06 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590682+0000", 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3596000000000006e-05 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.590708+0000", 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6089e-05 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593568+0000", 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028597789999999998 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:49.593614+0000", 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-05 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.5822048730000002, 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.1678914139999996, 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:27934 3.0 e50/23 3:06a5839e:::rbd_data.1176e641238f.0000000000000080:head v 50'87, mlcod=49'85)", 2024-09-18T01:08:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1469139549999996, 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030122230000000001, 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195656+0000", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5199999999999996e-07 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195662+0000", 2024-09-18T01:08:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8889999999999999e-06 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195673+0000", 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0913999999999999e-05 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195678+0000", 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1869999999999996e-06 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195733+0000", 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4197999999999998e-05 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.195755+0000", 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2790999999999999e-05 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.198622+0000", 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002866674 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.198668+0000", 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5818000000000001e-05 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:01.442 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:01.442 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:01.443 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:01.443 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:01.444 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:01.445 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:01.445 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:01.440+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:01.542 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:01.541+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:01.588 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:01.588 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:01.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:01.643 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:01.642+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:01.707 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:01.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:01.707 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:01.707 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:01.707 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:01.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:01.743 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:01.742+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:01.786 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8294 3.7 3:e3e34f1e:::rbd_data.11762d3aac1.0000000000000000:head [write 0~524288 [fadvise_sequential+fadvise_nocache],zero 524288~524288,write 1048576~524288 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~1048576 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.242032489, 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027761263000000001, 2024-09-18T01:08:01.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8294 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.584675+0000", 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586908+0000", 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022325259999999999 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586922+0000", 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4061e-05 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586931+0000", 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2089999999999994e-06 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587661+0000", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00073032299999999995 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587811+0000", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000150047 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.588205+0000", 2024-09-18T01:08:01.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039331099999999998 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610502+0000", 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022297437 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612252+0000", 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017498940000000001 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612391+0000", 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013893099999999999 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612436+0000", 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5093999999999999e-05 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8316 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 1048576~163840 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.835 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.239660861000001, 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012174538, 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8316 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587046+0000", 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587047+0000", 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0400000000000002e-07 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587049+0000", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.728e-06 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587055+0000", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9909999999999999e-06 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587060+0000", 2024-09-18T01:08:01.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5360000000000003e-06 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595805+0000", 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087449510000000008 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595925+0000", 2024-09-18T01:08:01.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012037200000000001 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599221+0000", 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032955559999999998 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8324 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 1572864~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.239354905999999, 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019081754999999999, 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8324 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:08:01.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587352+0000", 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587353+0000", 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2399999999999998e-07 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587356+0000", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9450000000000002e-06 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587363+0000", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.452e-06 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587369+0000", 2024-09-18T01:08:01.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5659999999999997e-06 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599368+0000", 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011999604000000001 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.599490+0000", 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121357 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606434+0000", 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006944207 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8341 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 0~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.239278409000001, 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02321252, 2024-09-18T01:08:01.845 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8341 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587429+0000", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587435+0000", 2024-09-18T01:08:01.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9610000000000002e-06 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587442+0000", 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7000000000000002e-06 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.587447+0000", 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606471+0000", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019024926000000001 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606525+0000", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3958000000000001e-05 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610641+0000", 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041158519999999997 2024-09-18T01:08:01.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8348 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.232822251000002, 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013699756, 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8348 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593885+0000", 2024-09-18T01:08:01.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593887+0000", 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2400000000000002e-06 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593902+0000", 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4421e-05 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593922+0000", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0020000000000001e-05 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.593929+0000", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7759999999999997e-06 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594130+0000", 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020075599999999999 2024-09-18T01:08:01.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594241+0000", 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001111 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594643+0000", 2024-09-18T01:08:01.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00040180700000000001 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600371+0000", 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057278600000000004 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607482+0000", 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071108730000000002 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607536+0000", 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4230000000000001e-05 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607585+0000", 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8673e-05 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8350 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.856 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.232422834000001, 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017945170999999999, 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8350 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594284+0000", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594285+0000", 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000004e-07 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594290+0000", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1710000000000001e-06 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594301+0000", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0499000000000001e-05 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594309+0000", 2024-09-18T01:08:01.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9650000000000008e-06 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610528+0000", 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016218713999999999 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610576+0000", 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8368999999999997e-05 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612230+0000", 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016536280000000001 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8390 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.230688068999999, 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01144247, 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.861 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8390 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596019+0000", 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596020+0000", 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596022+0000", 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8310000000000001e-06 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596029+0000", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6549999999999998e-06 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596044+0000", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4574e-05 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600514+0000", 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044701209999999996 2024-09-18T01:08:01.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600615+0000", 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100936 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607462+0000", 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068468139999999997 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8399 3.2 3:44ca0a04:::rbd_data.11762d3aac1.0000000000000006:head [write 0~3670016 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache],zero 5767168~524288,zero 7602176~262144,zero 15990784~218866,write 16209650~22755 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.212995299999999, 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029285142, 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8399 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.866 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613712+0000", 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613714+0000", 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9180000000000001e-06 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615140+0000", 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014266820000000001 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615157+0000", 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6330000000000001e-05 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615166+0000", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615300+0000", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013375400000000001 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615497+0000", 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019682799999999999 2024-09-18T01:08:01.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.616109+0000", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061267699999999995 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642827+0000", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026717366999999999 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642859+0000", 2024-09-18T01:08:01.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2777999999999997e-05 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642961+0000", 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101137 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.642997+0000", 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6449000000000003e-05 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8447 3.2 3:452cb511:::rbd_data.1176ec0d676b.0000000000000058:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.211323615000001, 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013050262, 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.872 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8447 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615384+0000", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615386+0000", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615392+0000", 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.037e-06 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615396+0000", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628305+0000", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012908974 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628434+0000", 2024-09-18T01:08:01.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012840200000000001 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8457 3.2 3:46e71cef:::rbd_data.1176ec0d676b.000000000000006a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.211282135000001, 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013141890999999999, 2024-09-18T01:08:01.876 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8457 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615425+0000", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615426+0000", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615428+0000", 2024-09-18T01:08:01.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.046e-06 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615433+0000", 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.615437+0000", 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9380000000000002e-06 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628468+0000", 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013030570999999999 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628567+0000", 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9035000000000003e-05 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:8397 3.5 e34/23 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head v 34'8, mlcod=31'6)", 2024-09-18T01:08:01.880 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.207760532999998, 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030584585000000001, 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.618947+0000", 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627937+0000", 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089904949999999994 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627956+0000", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8984999999999999e-05 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627964+0000", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5129999999999999e-06 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628036+0000", 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2004999999999999e-05 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.628066+0000", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9569000000000001e-05 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649485+0000", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021419779 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649531+0000", 2024-09-18T01:08:01.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5948999999999997e-05 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8785 3.2 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head [write 0~1146641 [fadvise_sequential+fadvise_nocache],zero 1146641~426223,write 1572864~2621440 [fadvise_sequential+fadvise_nocache],zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~557056 [fadvise_sequential+fadvise_nocache],zero 5259264~49152,write 5308416~81920 [fadvise_sequential+fadvise_nocache],zero 5390336~49152,write 5439488~81920 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.201259546999999, 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028310961999999999, 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.885 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8785 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625448+0000", 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625449+0000", 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.773e-06 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627661+0000", 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022114980000000001 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627688+0000", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6988e-05 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627697+0000", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8710000000000003e-06 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629596+0000", 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001899155 2024-09-18T01:08:01.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630212+0000", 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061557099999999998 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630882+0000", 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00067058700000000005 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649558+0000", 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018675605000000001 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653628+0000", 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004070097 2024-09-18T01:08:01.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653707+0000", 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9645000000000001e-05 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653759+0000", 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1171999999999997e-05 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8787 3.1e 3:7c1ba88b:::rbd_object_map.11762d3aac1.0000000000000008:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.194540996000001, 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021599920000000002, 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.892 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8787 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632166+0000", 2024-09-18T01:08:01.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632169+0000", 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632181+0000", 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1564e-05 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632203+0000", 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632227+0000", 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3484000000000001e-05 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632310+0000", 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2891000000000003e-05 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.632426+0000", 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116288 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635044+0000", 2024-09-18T01:08:01.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026176419999999999 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645349+0000", 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010305527 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653684+0000", 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083345120000000005 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653725+0000", 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1119999999999999e-05 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653766+0000", 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1287e-05 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8824 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.192948870999999, 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011546005, 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8824 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633758+0000", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633759+0000", 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:08:01.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633763+0000", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1189999999999999e-06 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633770+0000", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9129999999999997e-06 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.633775+0000", 2024-09-18T01:08:01.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645111+0000", 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.011336061999999999 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645226+0000", 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114784 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.645304+0000", 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8510999999999994e-05 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8960 3.7 3:e2b1a74c:::rbd_data.1176ec0d676b.0000000000000028:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.191989394, 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.014368275, 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.904 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8960 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634718+0000", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1200000000000003e-07 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634720+0000", 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:08:01.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634725+0000", 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.634735+0000", 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0176999999999999e-05 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643266+0000", 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0085303800000000006 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.643358+0000", 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1699000000000004e-05 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649086+0000", 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057285180000000002 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8965 3.7 3:e5b12c00:::rbd_data.1176ec0d676b.0000000000000003:7 [sparse-read 786432~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.191207909999999, 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015273905000000001, 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.909 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8965 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635499+0000", 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635500+0000", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-06 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635505+0000", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635516+0000", 2024-09-18T01:08:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1089e-05 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635529+0000", 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2754e-05 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649126+0000", 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013596964 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649214+0000", 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7905000000000006e-05 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.650773+0000", 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001559305 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8973 3.1e 3:7e62f05a:::rbd_data.1176ec0d676b.0000000000000012:7 [sparse-read 1048576~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.191165890000001, 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01383679, 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8973 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635541+0000", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635542+0000", 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9599999999999999e-07 2024-09-18T01:08:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635544+0000", 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4109999999999999e-06 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635551+0000", 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8109999999999997e-06 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635556+0000", 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8550000000000001e-06 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646171+0000", 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010615342 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.646218+0000", 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7259000000000003e-05 2024-09-18T01:08:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649378+0000", 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031596160000000001 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8982 3.1e 3:7ca2d17f:::rbd_data.1176ec0d676b.0000000000000022:7 [sparse-read 1310720~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.191144520000002, 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015554682, 2024-09-18T01:08:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8982 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635563+0000", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635565+0000", 2024-09-18T01:08:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8470000000000001e-06 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635571+0000", 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7219999999999996e-06 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.635575+0000", 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649410+0000", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.013835055000000001 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649481+0000", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0820000000000003e-05 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.651117+0000", 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016363770000000001 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9166 3.18 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.171562543, 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.028895934000000002, 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9166 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655145+0000", 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.655147+0000", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9879999999999999e-06 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656773+0000", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00162633 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656788+0000", 2024-09-18T01:08:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4547e-05 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656795+0000", 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2989999999999999e-06 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656857+0000", 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2203000000000001e-05 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.656994+0000", 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013699599999999999 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657438+0000", 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044429499999999999 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.682244+0000", 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024805437999999999 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683944+0000", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001699926 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683993+0000", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9447999999999998e-05 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.684041+0000", 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7463999999999998e-05 2024-09-18T01:08:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9173 3.19 e34/23 3:9b92acfd:::rbd_data.11762d3aac1.000000000000000e:head v 34'19, mlcod=31'17)", 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.159410753, 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.016520503999999998, 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667296+0000", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.667297+0000", 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3300000000000001e-07 2024-09-18T01:08:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668283+0000", 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098571899999999992 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668305+0000", 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1665999999999999e-05 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668317+0000", 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2013e-05 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668402+0000", 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4970999999999997e-05 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.668429+0000", 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7416e-05 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683769+0000", 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015339999999999999 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683817+0000", 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7886000000000002e-05 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:01.935 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:01.935 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:01.934+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:02.035 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:02.035+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:02.041 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:02.041 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.050 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:02.061 INFO:teuthology.orchestra.run.smithi120.stdout:787 trunc from 0xbec4dc1 to 0xce133b 2024-09-18T01:08:02.082 INFO:teuthology.orchestra.run.smithi120.stdout:788 punch from 0x26f34d to 0x27c032, (0xcce5 bytes) 2024-09-18T01:08:02.082 INFO:teuthology.orchestra.run.smithi120.stdout:789 read 0x179c2f thru 0x1851d7 (0xb5a9 bytes) 2024-09-18T01:08:02.082 INFO:teuthology.orchestra.run.smithi120.stdout:790 read 0x5573ac thru 0x563b1d (0xc772 bytes) 2024-09-18T01:08:02.086 INFO:teuthology.orchestra.run.smithi120.stdout:792 writesame 0x1999f23 thru 0x19a1a18 data_size 0x1(0x7af6 bytes) 2024-09-18T01:08:02.095 INFO:teuthology.orchestra.run.smithi120.stdout:793 trunc from 0x19a1a19 to 0x8796217 2024-09-18T01:08:02.101 INFO:teuthology.orchestra.run.smithi120.stdout:794 punch from 0x8655bbd to 0x86631fc, (0xd63f bytes) 2024-09-18T01:08:02.101 INFO:teuthology.orchestra.run.smithi120.stdout:795 read 0x8f1986 thru 0x8fc533 (0xabae bytes) 2024-09-18T01:08:02.102 INFO:teuthology.orchestra.run.smithi120.stdout:798 punch from 0xde7ac8 to 0xdedadb, (0x6013 bytes) 2024-09-18T01:08:02.102 INFO:teuthology.orchestra.run.smithi120.stdout:799 read 0x5a8c64e thru 0x5a96966 (0xa319 bytes) 2024-09-18T01:08:02.102 INFO:teuthology.orchestra.run.smithi120.stdout:800 clone 10 order 25 su 4194304 sc 6 2024-09-18T01:08:02.136 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:02.136+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:02.155 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:02.155 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.164 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:02.237 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:02.236+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.675575375000001, 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:02.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:02.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.667243703, 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.663902848999999, 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:02.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.63644802, 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.284 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:02.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.602482703, 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:02.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:02.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.521515117, 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:02.298 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.521493279000001, 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:02.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:02.303 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6311479789999996, 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:02.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:02.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17763 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 98304~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6296314719999998, 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092008709999999994, 2024-09-18T01:08:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17763 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637404+0000", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637405+0000", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637408+0000", 2024-09-18T01:08:02.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.574e-06 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637416+0000", 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2450000000000001e-06 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637422+0000", 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3820000000000001e-06 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.644983+0000", 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0075605960000000002 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.645035+0000", 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1925000000000002e-05 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646605+0000", 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015706019999999999 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.312 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6296051140000003, 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:02.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.317 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6295839569999995, 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.318 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:02.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:02.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6295651430000007, 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:02.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:02.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1760583469999997, 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:08:02.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:08:02.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1760469669999996, 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:02.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:02.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1760330469999998, 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:02.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28135 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.8883249800000002, 2024-09-18T01:08:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095049399999999999, 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28135 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378711+0000", 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6e-07 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378713+0000", 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.699e-06 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378716+0000", 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.507e-06 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.378719+0000", 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6170000000000001e-06 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.381793+0000", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030739780000000002 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.381869+0000", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6171e-05 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.382251+0000", 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038233000000000001 2024-09-18T01:08:02.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.387988+0000", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057363559999999997 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388151+0000", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163695 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388189+0000", 2024-09-18T01:08:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7999e-05 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.388216+0000", 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6327999999999999e-05 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:02.351 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6994984620000002, 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:08:02.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:02.357 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6994869829999999, 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:02.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:02.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.5442951239999996, 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:02.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:02.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:02.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.6072891249999999, 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:02.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.377 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.377 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:02.379 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:02.377+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:02.479 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:02.478+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:02.482 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:02.482 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.490 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:02.556 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:08:02.593 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:02.594 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.601 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:02.600+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:02.602 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:02.652 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:02.652 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:02.652 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:08:02.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:08:02.680 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:02.679+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:02.711 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686449471, 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686407320000001, 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686373891999999, 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686336392000001, 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686314814999999, 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:02.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686295675, 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:02.735 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686275284000001, 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686245884000002, 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:02.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:02.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:02.749 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686217094, 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:02.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686196399, 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:02.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:02.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686176915000001, 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:02.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:02.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.764 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686143004000002, 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:02.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:02.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.686107005, 2024-09-18T01:08:02.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:02.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:02.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.601659033000001, 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:02.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.601648824000002, 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:02.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:02.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.101247598, 2024-09-18T01:08:02.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.085761429000001, 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.077870708999999, 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.051801705999999, 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.046632914, 2024-09-18T01:08:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:02.816 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:02.818 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:02.818 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:02.817+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:02.919 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:02.918+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 56, 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:08:02.648052+0000", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:07:01.953049+0000", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5799936, 2024-09-18T01:08:02.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:08:02.943 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "jewel", 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:02.944 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:02.945 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:02.946 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:02.947 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:02.948 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.949 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 6.9000000953674316, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 6.9000000953674316, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.28999999165534973, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:08:02.950 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:02.951 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:02.952 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 2.630000114440918, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 2.630000114440918, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2.630000114440918, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2.630000114440918, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:02.957 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:02.958 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "56", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 21, 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 56, 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:02.959 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:02.960 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:02.961 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5399999618530273, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5399999618530273, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1.1399999856948853, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:08:02.962 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.963 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.968 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:02.969 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:08:02.971 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.972 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.973 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 18, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.975 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.976 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:08:02.980 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.981 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.982 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:08:02.985 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.986 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.987 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.991 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.992 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:02.997 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 0, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:02.998 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:08:02.999 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.000 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:03.004 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.005 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.006 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:03.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:03.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:03.011 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.012 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.013 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:08:03.014 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:08:03.017 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [], 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:08:03.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 16, 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 18, 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:03.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 20, 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:08:03.020 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:03.021 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:03.023 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:03.023 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:03.023 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:03.023 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:03.024 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:08:02.648052+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:07:01.953049+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 1}, {'begin': 18, 'length': 1}, {'begin': 20, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:08:03.024 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:08:03.024 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:03.021+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:03.025 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:03.123 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:03.122+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:03.128 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:03.129 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:03.137 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:03.225 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:03.224+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:03.241 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.157611943999999, 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.054297647999999, 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:03.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.577621095000001, 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:03.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:03.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.493688856999999, 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:03.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.491679724000001, 2024-09-18T01:08:03.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:03.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:03.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.270 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13185 3.a 3:51a12e8c:::rbd_data.117611b60058.00000000000000ee:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.090976174000001, 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00605274, 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13185 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148174+0000", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148179+0000", 2024-09-18T01:08:03.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148182+0000", 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2540000000000001e-06 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148734+0000", 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055139499999999999 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148870+0000", 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013613699999999999 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.154225+0000", 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053546829999999998 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13241 3.10 3:090d2a4d:::rbd_data.117611b60058.0000000000000149:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.067253503, 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005514878, 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13241 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171896+0000", 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171908+0000", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2401e-05 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171918+0000", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.042e-05 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171925+0000", 2024-09-18T01:08:03.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171967+0000", 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1257000000000002e-05 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.172085+0000", 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011883999999999999 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.177409+0000", 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053241210000000002 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:15123 3.1e e43/23 3:7a2bedf1:::rbd_data.1176492bc9ca.0000000000000000:head v 43'60, mlcod=42'58)", 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.674269664000001, 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061319520000000004, 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564879+0000", 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565612+0000", 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073299200000000002 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565627+0000", 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5634e-05 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565638+0000", 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0462000000000001e-05 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565707+0000", 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9402999999999999e-05 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565730+0000", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3108999999999998e-05 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570965+0000", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052347380000000001 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571010+0000", 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5187e-05 2024-09-18T01:08:03.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2604559989999995, 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:03.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:03.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18543 3.a 3:5239904c:::rbd_data.11762d95e74a.0000000000000038:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1996268580000002, 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059139489999999999, 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18543 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039523+0000", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.325e-06 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039529+0000", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039541+0000", 2024-09-18T01:08:03.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2342e-05 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039548+0000", 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.99e-06 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039592+0000", 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3915000000000002e-05 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039646+0000", 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3668000000000002e-05 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.040432+0000", 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000786288 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.044920+0000", 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004487624 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045325+0000", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000405308 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045381+0000", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5550000000000002e-05 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045435+0000", 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4657000000000002e-05 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27452 3.a 3:52657a03:::rbd_data.1176492bc9ca.0000000000000045:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.34047026, 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057803079999999996, 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27452 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898679+0000", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898682+0000", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8060000000000001e-06 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898694+0000", 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1652e-05 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898700+0000", 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4969999999999998e-06 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898807+0000", 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010601 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:03.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898813+0000", 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904378+0000", 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055656710000000003 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904458+0000", 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9935e-05 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27469 3.a 3:52bb01ff:::rbd_data.1176492bc9ca.0000000000000056:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.3404289729999999, 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058442279999999999, 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27469 2024-09-18T01:08:03.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898720+0000", 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.37e-07 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898723+0000", 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898730+0000", 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898735+0000", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898843+0000", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010823900000000001 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898848+0000", 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8230000000000002e-06 2024-09-18T01:08:03.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904484+0000", 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056357589999999997 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904563+0000", 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9746000000000005e-05 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27543 3.a 3:57ca41ae:::rbd_data.1176492bc9ca.00000000000000a0:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.3399606349999997, 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054992519999999996, 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27543 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899188+0000", 2024-09-18T01:08:03.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899197+0000", 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2720000000000003e-06 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899205+0000", 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5850000000000002e-06 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899211+0000", 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509999999999997e-06 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899308+0000", 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7405999999999995e-05 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899313+0000", 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0139999999999998e-06 2024-09-18T01:08:03.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904601+0000", 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005288292 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904687+0000", 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499000000000003e-05 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27546 3.a 3:5585dcf7:::rbd_data.1176492bc9ca.00000000000000a3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.3399245899999999, 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055778030000000001, 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27546 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:03.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899226+0000", 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5000000000000002e-06 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899234+0000", 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1679999999999998e-06 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899239+0000", 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999998e-06 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899338+0000", 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.912e-05 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899344+0000", 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904713+0000", 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053690400000000003 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904801+0000", 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7967999999999995e-05 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.316 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27837 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0897122350000004, 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056104429999999997, 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27837 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149437+0000", 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999997e-07 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149449+0000", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2368000000000001e-05 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149462+0000", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3277e-05 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149471+0000", 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3799999999999994e-06 2024-09-18T01:08:03.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149517+0000", 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5838999999999999e-05 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149607+0000", 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0650999999999997e-05 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149715+0000", 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107345 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154757+0000", 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005042571 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154948+0000", 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019090200000000001 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155005+0000", 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6504e-05 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155046+0000", 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1696000000000002e-05 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28147 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.8421728020000003, 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006127146, 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28147 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396977+0000", 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4050000000000001e-06 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396988+0000", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1046000000000001e-05 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397002+0000", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3895e-05 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397008+0000", 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:08:03.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397064+0000", 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5961999999999997e-05 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397147+0000", 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3233999999999996e-05 2024-09-18T01:08:03.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397295+0000", 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014745700000000001 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.400032+0000", 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027374460000000001 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403019+0000", 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029870600000000001 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403061+0000", 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1999000000000002e-05 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403102+0000", 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1072999999999998e-05 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.328 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.733385073, 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:03.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:03.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.733357292, 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:03.335 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:03.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:03.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.731938199, 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:03.341 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:03.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:03.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:03.347 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6038760080000003, 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:03.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:03.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:03.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:03.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:03.352 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:03.353 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:03.354 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:03.352+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:03.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:03.490 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:03.489+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:03.496 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:03.496 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:03.497 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:03.497 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:03.505 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:03.554 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:03.553+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:03.566 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:03.567 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:03.583 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:03.583 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:03.599 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:03.599 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:03.608 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:03.608 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:03.608 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:03.608 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:03.608 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:03.615 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:03.615 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:03.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:03.632 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:03.633 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:03.648 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:03.648 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:03.655 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:03.654+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:03.665 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:03.666 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:03.666 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:08:03.692 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:03.693 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:03.756 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:03.755+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:03.773 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:03.774 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:03.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:03.857 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:03.856+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:03.886 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:03.886 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:03.895 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:03.957 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:03.957+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:03.997 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:03.997 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:04.058 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:04.058+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:04.109 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.109 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:04.159 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:04.158+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:04.260 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:04.259+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:04.261 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.261 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:04.361 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:04.360+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:04.372 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.372 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.381 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:04.466 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:04.465+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:04.524 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.524 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.533 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:04.574 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:04.573+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:04.658 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-18T01:08:04.664 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:04.663+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:04.687 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.687 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.699 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:04.765 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:04.764+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:04.815 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.815 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.824 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:04.866 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:04.865+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:04.884 INFO:teuthology.orchestra.run.smithi120.stdout:801 read 0x63989ff thru 0x639eae8 (0x60ea bytes) 2024-09-18T01:08:04.885 INFO:teuthology.orchestra.run.smithi120.stdout:802 punch from 0x5e5c611 to 0x5e679d3, (0xb3c2 bytes) 2024-09-18T01:08:04.897 INFO:teuthology.orchestra.run.smithi120.stdout:803 write 0xba7c867 thru 0xba89832 (0xcfcc bytes) 2024-09-18T01:08:04.902 INFO:teuthology.orchestra.run.smithi120.stdout:805 write 0xcfe9f2 thru 0xd07a4d (0x905c bytes) 2024-09-18T01:08:04.905 INFO:teuthology.orchestra.run.smithi120.stdout:806 read 0x5ffbb7f thru 0x6000a37 (0x4eb9 bytes) 2024-09-18T01:08:04.905 INFO:teuthology.orchestra.run.smithi120.stdout:807 write 0x207ab57 thru 0x207c3f6 (0x18a0 bytes) 2024-09-18T01:08:04.906 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:04.906 INFO:teuthology.orchestra.run.smithi120.stdout:809 clone 11 order 25 su 4194304 sc 5 2024-09-18T01:08:04.967 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:04.966+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:04.969 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:04.969 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:04.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:05.068 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:05.067+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:05.121 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.121 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:05.171 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:05.170+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:05.273 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.273 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.273 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:05.272+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:05.282 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:05.374 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:05.373+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:05.385 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.385 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.394 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:05.477 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:05.476+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:05.504 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.505 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.514 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:05.577 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:05.576+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:05.618 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.618 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.626 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:05.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:08:05.705 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:05.711 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:05.710+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:05.772 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.772 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.779 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:05.779+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:05.781 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:05.852 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:05.853 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:05.869 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:05.869 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:05.881 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:05.880+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:05.885 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:05.888 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:05.901 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:05.917 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:05.918 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:05.925 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:05.925 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:05.934 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:05.934 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:05.935 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:05.950 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:05.950 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:05.967 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:05.967 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:05.981 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:05.980+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:06.037 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.037 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:06.037 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:06.037 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:06.038 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.046 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:06.083 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:06.082+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:06.184 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:06.183+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:06.191 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166626161, 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166584010000001, 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166550581999999, 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166513082000002, 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166491505, 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166472365000001, 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166451974000001, 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166422573999998, 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166393784, 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166373089, 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166353605000001, 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166319693999998, 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.166283695000001, 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.081835723000001, 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.081825513999998, 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.581424288000001, 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.565938118999998, 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.558047398999999, 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.531978396, 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.526809604, 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.287 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:06.289 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:06.287+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:06.389 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:06.388+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.401 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:06.490 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:06.489+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:06.504 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.504 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:06.504 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:06.504 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:06.504 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.513 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:06.591 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:06.590+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:06.618 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.535077483999999, 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.619 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:06.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:06.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:06.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:06.626 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.431763188000001, 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:06.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:06.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:06.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.955086635000001, 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:06.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.871154397000002, 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:06.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.646 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.869145264, 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.647 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:06.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13185 3.a 3:51a12e8c:::rbd_data.117611b60058.00000000000000ee:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.468441714000001, 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00605274, 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13185 2024-09-18T01:08:06.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148174+0000", 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148179+0000", 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148182+0000", 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2540000000000001e-06 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148734+0000", 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055139499999999999 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148870+0000", 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013613699999999999 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.154225+0000", 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053546829999999998 2024-09-18T01:08:06.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13241 3.10 3:090d2a4d:::rbd_data.117611b60058.0000000000000149:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.444719042999999, 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005514878, 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:06.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13241 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171896+0000", 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171908+0000", 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2401e-05 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171918+0000", 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.042e-05 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171925+0000", 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:08:06.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171967+0000", 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1257000000000002e-05 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.172085+0000", 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011883999999999999 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.177409+0000", 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053241210000000002 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.663 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:15123 3.1e e43/23 3:7a2bedf1:::rbd_data.1176492bc9ca.0000000000000000:head v 43'60, mlcod=42'58)", 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.051735204, 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061319520000000004, 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:06.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564879+0000", 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565612+0000", 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073299200000000002 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565627+0000", 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5634e-05 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565638+0000", 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0462000000000001e-05 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565707+0000", 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9402999999999999e-05 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565730+0000", 2024-09-18T01:08:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3108999999999998e-05 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570965+0000", 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052347380000000001 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571010+0000", 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5187e-05 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.668 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.637921539000001, 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:06.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:06.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18543 3.a 3:5239904c:::rbd_data.11762d95e74a.0000000000000038:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.577092398, 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059139489999999999, 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18543 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039523+0000", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.325e-06 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039529+0000", 2024-09-18T01:08:06.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039541+0000", 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2342e-05 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039548+0000", 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.99e-06 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039592+0000", 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3915000000000002e-05 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039646+0000", 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3668000000000002e-05 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.040432+0000", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000786288 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.044920+0000", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004487624 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045325+0000", 2024-09-18T01:08:06.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000405308 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045381+0000", 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5550000000000002e-05 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045435+0000", 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4657000000000002e-05 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27452 3.a 3:52657a03:::rbd_data.1176492bc9ca.0000000000000045:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7179357999999993, 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057803079999999996, 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27452 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898679+0000", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898682+0000", 2024-09-18T01:08:06.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8060000000000001e-06 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898694+0000", 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1652e-05 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898700+0000", 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4969999999999998e-06 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898807+0000", 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010601 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898813+0000", 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904378+0000", 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055656710000000003 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904458+0000", 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9935e-05 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27469 3.a 3:52bb01ff:::rbd_data.1176492bc9ca.0000000000000056:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7178945129999992, 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058442279999999999, 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27469 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898720+0000", 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.37e-07 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898723+0000", 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898730+0000", 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898735+0000", 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898843+0000", 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010823900000000001 2024-09-18T01:08:06.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898848+0000", 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8230000000000002e-06 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904484+0000", 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056357589999999997 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904563+0000", 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9746000000000005e-05 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27543 3.a 3:57ca41ae:::rbd_data.1176492bc9ca.00000000000000a0:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7174261749999999, 2024-09-18T01:08:06.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054992519999999996, 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27543 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899187+0000", 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899188+0000", 2024-09-18T01:08:06.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899197+0000", 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2720000000000003e-06 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899205+0000", 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5850000000000002e-06 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899211+0000", 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509999999999997e-06 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899308+0000", 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7405999999999995e-05 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899313+0000", 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0139999999999998e-06 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904601+0000", 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005288292 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904687+0000", 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499000000000003e-05 2024-09-18T01:08:06.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27546 3.a 3:5585dcf7:::rbd_data.1176492bc9ca.00000000000000a3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7173901300000001, 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055778030000000001, 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.700 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27546 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:06.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899226+0000", 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5000000000000002e-06 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899234+0000", 2024-09-18T01:08:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1679999999999998e-06 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899239+0000", 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999998e-06 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899338+0000", 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.912e-05 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899344+0000", 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904713+0000", 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053690400000000003 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904801+0000", 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7967999999999995e-05 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27837 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4671777749999997, 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056104429999999997, 2024-09-18T01:08:06.707 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27837 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:06.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149437+0000", 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999997e-07 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149449+0000", 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2368000000000001e-05 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149462+0000", 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3277e-05 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149471+0000", 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3799999999999994e-06 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149517+0000", 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5838999999999999e-05 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149607+0000", 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0650999999999997e-05 2024-09-18T01:08:06.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149715+0000", 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107345 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154757+0000", 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005042571 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154948+0000", 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019090200000000001 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155005+0000", 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6504e-05 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155046+0000", 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1696000000000002e-05 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28147 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2196383419999997, 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006127146, 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28147 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396977+0000", 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4050000000000001e-06 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396988+0000", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1046000000000001e-05 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397002+0000", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3895e-05 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397008+0000", 2024-09-18T01:08:06.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397064+0000", 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5961999999999997e-05 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397147+0000", 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3233999999999996e-05 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397295+0000", 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014745700000000001 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.400032+0000", 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027374460000000001 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403019+0000", 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029870600000000001 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403061+0000", 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1999000000000002e-05 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403102+0000", 2024-09-18T01:08:06.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1072999999999998e-05 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1108506130000002, 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.721 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:06.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:06.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1108228320000002, 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:06.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:06.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:06.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1094037389999993, 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:06.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:06.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.9813415479999996, 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:06.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:06.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:06.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.749 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:08:06.750 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:06.750 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:06.747+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:06.850 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:06.848+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:06.892 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:06.893 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:06.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:06.985 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:06.984+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:07.044 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:07.045 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:07.045 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:07.045 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:07.045 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:07.051 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:07.050+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:07.053 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:07.152 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:07.151+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:07.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 55.300146619000003, 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:07.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:07.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3015 3.15 3:ac0b78c0:::rbd_data.1176f5693ee0.000000000000003a:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.907020146000001, 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012252512, 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.207 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3015 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290935+0000", 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290936+0000", 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290938+0000", 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290949+0000", 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1538e-05 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290954+0000", 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8919999999999999e-06 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300757+0000", 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098030359999999993 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300798+0000", 2024-09-18T01:08:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0899999999999998e-05 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.303188+0000", 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002389689 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.906967321, 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.540236020999998, 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12807 3.d 3:b5154276:::rbd_data.1176ec0d676b.0000000000000016:head [sparse-read 0~1252609] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.451187327, 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011900506999999999, 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12807 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746768+0000", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746769+0000", 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2100000000000001e-06 2024-09-18T01:08:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746779+0000", 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0023e-05 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746789+0000", 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0105000000000001e-05 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746803+0000", 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3784999999999999e-05 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753526+0000", 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067230479999999997 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753625+0000", 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8485000000000003e-05 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758668+0000", 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050438510000000002 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12810 3.15 3:ae5c894f:::rbd_data.1176ec0d676b.0000000000000019:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.451121284999999, 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01205553, 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12810 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746834+0000", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1300000000000001e-07 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746837+0000", 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3269999999999999e-06 2024-09-18T01:08:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746846+0000", 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6690000000000001e-06 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746850+0000", 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8310000000000003e-06 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753576+0000", 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0067258439999999999 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753682+0000", 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105919 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758890+0000", 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005207527 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12826 3.d 3:b63623b7:::rbd_data.1176ec0d676b.0000000000000029:head [sparse-read 0~1044659] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.451084151, 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014765014999999999, 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12826 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746872+0000", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746874+0000", 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:08:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746884+0000", 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0088999999999999e-05 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746889+0000", 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.259e-06 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758702+0000", 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011813259 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758789+0000", 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6386999999999998e-05 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761636+0000", 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002847488 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.260848872, 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15117 3.e 3:7282a290:::rbd_object_map.1176492bc9ca.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.639038930000002, 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012024517, 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15117 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558916+0000", 2024-09-18T01:08:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558917+0000", 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499999999999997e-07 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558923+0000", 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558935+0000", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2143e-05 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.558941+0000", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7349999999999999e-06 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561319+0000", 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023776959999999999 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561407+0000", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8103999999999995e-05 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.561647+0000", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239454 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570808+0000", 2024-09-18T01:08:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0091611109999999996 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570822+0000", 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3788000000000001e-05 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570886+0000", 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3975999999999999e-05 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570941+0000", 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5313999999999999e-05 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:15128 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.626459654000001, 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011752311999999999, 2024-09-18T01:08:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15128 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571496+0000", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9100000000000003e-07 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572019+0000", 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052237400000000004 2024-09-18T01:08:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572034+0000", 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4963999999999999e-05 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572041+0000", 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9580000000000001e-06 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572094+0000", 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3115999999999998e-05 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572230+0000", 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013626 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.572378+0000", 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014820799999999999 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.577599+0000", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052203060000000001 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583147+0000", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055480829999999997 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583211+0000", 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4766999999999998e-05 2024-09-18T01:08:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.583248+0000", 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6485000000000001e-05 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.568527147999999, 2024-09-18T01:08:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.568512051000001, 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.568500901, 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.568491264, 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.564579216, 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17718 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.563545960999999, 2024-09-18T01:08:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015103220000000001, 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17718 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634413+0000", 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634417+0000", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5689999999999998e-06 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634429+0000", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2082000000000001e-05 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634445+0000", 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5818e-05 2024-09-18T01:08:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649397+0000", 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014951391 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649457+0000", 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0001999999999999e-05 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649513+0000", 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066999999999997e-05 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17726 3.f 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.634733+0000", 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.563221879, 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012679404, 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17726 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634733+0000", 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634733+0000", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634734+0000", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5400000000000001e-07 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635420+0000", 2024-09-18T01:08:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068566500000000002 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635440+0000", 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.997e-05 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635451+0000", 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.117e-05 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635514+0000", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3188000000000006e-05 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635650+0000", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013560800000000001 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635783+0000", 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013363900000000001 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647069+0000", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011285575000000001 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647300+0000", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023097800000000001 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647357+0000", 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7138000000000001e-05 2024-09-18T01:08:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647413+0000", 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5819000000000002e-05 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17734 3.12 e46/23 3:4cb2c02c:::rbd_data.11762d95e74a.0000000000000002:head v 46'79, mlcod=44'77)", 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.561616065000001, 2024-09-18T01:08:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014297084999999999, 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636340+0000", 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636751+0000", 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041088000000000002 2024-09-18T01:08:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636760+0000", 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1640000000000007e-06 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636767+0000", 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0219999999999999e-06 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636823+0000", 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6632000000000003e-05 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636844+0000", 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.056e-05 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650572+0000", 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013728290000000001 2024-09-18T01:08:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650636+0000", 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4144000000000001e-05 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.50107796599999999, 2024-09-18T01:08:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:38007 3.12 e60/23 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:head v 60'103, mlcod=51'101)", 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.493952326, 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015053626000000001, 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6399999999999998e-07 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711136+0000", 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071328030000000001 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711149+0000", 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3047000000000001e-05 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711157+0000", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711210+0000", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2818999999999997e-05 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711227+0000", 2024-09-18T01:08:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7688000000000001e-05 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.719021+0000", 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077937309999999999 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.719057+0000", 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5644e-05 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:07.313 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:07.314 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:07.315 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:07.313+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:07.415 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:07.414+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:07.423 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:07.423 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:07.424 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:07.424 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:07.432 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:07.517 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:07.516+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:07.575 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:07.576 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:07.576 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:07.576 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:07.576 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:07.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:07.616 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:07.616+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:07.717 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:07.716+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1066 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664144665999999, 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017957565000000002, 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1066 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068800+0000", 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068802+0000", 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068808+0000", 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8470000000000001e-06 2024-09-18T01:08:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068812+0000", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1910000000000002e-06 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085836+0000", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017023470999999998 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.085904+0000", 2024-09-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8657000000000004e-05 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086757+0000", 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085298500000000001 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1067 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:08:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664125742000003, 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017422337, 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1067 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068819+0000", 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068821+0000", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1849999999999998e-06 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068827+0000", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0889999999999996e-06 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068832+0000", 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4170000000000002e-06 2024-09-18T01:08:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086073+0000", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017241587999999999 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086142+0000", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8508000000000002e-05 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086241+0000", 2024-09-18T01:08:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9174999999999998e-05 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1068 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664106502000003, 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017590855999999998, 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1068 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:08:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068838+0000", 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068840+0000", 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.939e-06 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068846+0000", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8719999999999999e-06 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068851+0000", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4920000000000004e-06 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086267+0000", 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017416338 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086343+0000", 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6082999999999996e-05 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086429+0000", 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5807e-05 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1069 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664087868000003, 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017961016999999999, 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1069 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068857+0000", 2024-09-18T01:08:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068859+0000", 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068865+0000", 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3249999999999996e-06 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068871+0000", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8749999999999997e-06 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086452+0000", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017580423000000001 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086716+0000", 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026484300000000001 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086818+0000", 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010118199999999999 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1070 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664067488000001, 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018106271, 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1070 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068877+0000", 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068884+0000", 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6499999999999999e-06 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068890+0000", 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3239999999999999e-06 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068895+0000", 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9280000000000001e-06 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086839+0000", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.017944281999999999 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086905+0000", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6000000000000005e-05 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.086983+0000", 2024-09-18T01:08:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7736000000000002e-05 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1071 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664042004999999, 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018256174, 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1071 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068902+0000", 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068903+0000", 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068905+0000", 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3800000000000001e-06 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068911+0000", 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8680000000000004e-06 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068916+0000", 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5900000000000001e-06 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087005+0000", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018089085000000001 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087072+0000", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7306999999999998e-05 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087159+0000", 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6496999999999995e-05 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1072 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.664022416999998, 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018416670999999999, 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1072 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:08:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068922+0000", 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068924+0000", 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068940+0000", 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5594999999999999e-05 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068946+0000", 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087184+0000", 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018238134999999999 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087253+0000", 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9287000000000003e-05 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087339+0000", 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5439999999999995e-05 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:1073 3.1b 3:da9a4188:::rbd_data.1176f5693ee0.0000000000000004:3 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.663982185000002, 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018558742999999999, 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1073 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068962+0000", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068963+0000", 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-07 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068965+0000", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068972+0000", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9120000000000001e-06 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.068977+0000", 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7679999999999999e-06 2024-09-18T01:08:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087363+0000", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018385865000000001 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087437+0000", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3930999999999994e-05 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.087521+0000", 2024-09-18T01:08:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4030999999999996e-05 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.146381773000002, 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.142748019999999, 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.142724452, 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:07.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.141149049999999, 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:07.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.136715959, 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:07.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:07.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.135393782000001, 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:07.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.131339970999999, 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:07.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.105651542, 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:07.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:07.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.075113337000001, 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:07.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:07.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:07.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:07.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.074623644999999, 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:07.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:07.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:07.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.010263949, 2024-09-18T01:08:07.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:07.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:07.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:07.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38007 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:head [write 0~3544682 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:06.700668+0000", 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.0322759640000001, 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018632098, 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:07.835 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38007 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.700668+0000", 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:07.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.700668+0000", 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.700670+0000", 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3990000000000001e-06 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703268+0000", 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025977679999999999 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703288+0000", 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9890000000000001e-05 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703308+0000", 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0806999999999999e-05 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703413+0000", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104274 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703554+0000", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000141333 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.703688+0000", 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000134077 2024-09-18T01:08:07.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.713919+0000", 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010230776 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.719220+0000", 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053012570000000002 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:07.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.719263+0000", 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2617999999999998e-05 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:06.719301+0000", 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7898999999999997e-05 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:07.841 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:07.843 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:07.843 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-18T01:08:07.843 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:07.842+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:07.943 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:07.942+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:07.971 INFO:teuthology.orchestra.run.smithi120.stdout:811 write 0x6644e9f thru 0x664f91c (0xaa7e bytes) 2024-09-18T01:08:07.977 INFO:teuthology.orchestra.run.smithi120.stdout:813 punch from 0x9fda305 to 0x9fde7ff, (0x44fa bytes) 2024-09-18T01:08:07.977 INFO:teuthology.orchestra.run.smithi120.stdout:815 read 0x4dfac2c thru 0x4e00a13 (0x5de8 bytes) 2024-09-18T01:08:07.977 INFO:teuthology.orchestra.run.smithi120.stdout:816 read 0x911f783 thru 0x912181f (0x209d bytes) 2024-09-18T01:08:07.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:07.979 INFO:teuthology.orchestra.run.smithi120.stdout:817 punch from 0x94953cb to 0x949c697, (0x72cc bytes) 2024-09-18T01:08:07.979 INFO:teuthology.orchestra.run.smithi120.stdout:818 punch from 0xb5439e6 to 0xb54585b, (0x1e75 bytes) 2024-09-18T01:08:07.979 INFO:teuthology.orchestra.run.smithi120.stdout:819 read 0x246b735 thru 0x24788d6 (0xd1a2 bytes) 2024-09-18T01:08:07.979 INFO:teuthology.orchestra.run.smithi120.stdout:820 write 0x9f76345 thru 0x9f7b50a (0x51c6 bytes) 2024-09-18T01:08:07.981 INFO:teuthology.orchestra.run.smithi120.stdout:821 writesame 0x3d85a76 thru 0x3d8c4d2 data_size 0x1(0x6a5d bytes) 2024-09-18T01:08:07.996 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:07.997 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:07.997 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:07.997 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:07.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:07.997 INFO:teuthology.orchestra.run.smithi120.stdout:824 write 0x9ac6173 thru 0x9ad2b74 (0xca02 bytes) 2024-09-18T01:08:08.002 INFO:teuthology.orchestra.run.smithi120.stdout:825 write 0x9292b59 thru 0x929d3a0 (0xa848 bytes) 2024-09-18T01:08:08.004 INFO:teuthology.orchestra.run.smithi120.stdout:827 write 0x54d0a0 thru 0x55781e (0xa77f bytes) 2024-09-18T01:08:08.006 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.006 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:08.012 INFO:teuthology.orchestra.run.smithi120.stdout:830 writesame 0xc4a5878 thru 0xc4ac71e data_size 0x1(0x6ea7 bytes) 2024-09-18T01:08:08.012 INFO:teuthology.orchestra.run.smithi120.stdout:833 punch from 0x8fda437 to 0x8fea3f6, (0xffbf bytes) 2024-09-18T01:08:08.012 INFO:teuthology.orchestra.run.smithi120.stdout:834 read 0x8bc6e34 thru 0x8bc8de3 (0x1fb0 bytes) 2024-09-18T01:08:08.013 INFO:teuthology.orchestra.run.smithi120.stdout:835 write 0x9a38df0 thru 0x9a41d05 (0x8f16 bytes) 2024-09-18T01:08:08.014 INFO:teuthology.orchestra.run.smithi120.stdout:836 writesame 0x7eeca3a thru 0x7eee2f5 data_size 0x2(0x18bc bytes) 2024-09-18T01:08:08.017 INFO:teuthology.orchestra.run.smithi120.stdout:837 punch from 0x39e8970 to 0x39f874d, (0xfddd bytes) 2024-09-18T01:08:08.017 INFO:teuthology.orchestra.run.smithi120.stdout:838 read 0x261c5b6 thru 0x2620214 (0x3c5f bytes) 2024-09-18T01:08:08.017 INFO:teuthology.orchestra.run.smithi120.stdout:839 write 0x2a5909f thru 0x2a65bfb (0xcb5d bytes) 2024-09-18T01:08:08.019 INFO:teuthology.orchestra.run.smithi120.stdout:840 punch from 0x6c56e0e to 0x6c5def0, (0x70e2 bytes) 2024-09-18T01:08:08.019 INFO:teuthology.orchestra.run.smithi120.stdout:841 read 0xa7c86d8 thru 0xa7cb5a3 (0x2ecc bytes) 2024-09-18T01:08:08.019 INFO:teuthology.orchestra.run.smithi120.stdout:842 read 0x1f44e03 thru 0x1f464ec (0x16ea bytes) 2024-09-18T01:08:08.020 INFO:teuthology.orchestra.run.smithi120.stdout:843 read 0x48a7c1e thru 0x48af019 (0x73fc bytes) 2024-09-18T01:08:08.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:08.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:08.025 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:08:08.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:08:08.033 INFO:teuthology.orchestra.run.smithi120.stdout:844 trunc from 0xc4ac71f to 0x304ffe7 2024-09-18T01:08:08.044 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:08.043+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:08.069 INFO:teuthology.orchestra.run.smithi120.stdout:846 writesame 0xd81b043 thru 0xd81b1f6 data_size 0x2(0x1b4 bytes) 2024-09-18T01:08:08.069 INFO:teuthology.orchestra.run.smithi120.stdout:847 read 0x8d7e058 thru 0x8d81147 (0x30f0 bytes) 2024-09-18T01:08:08.069 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.069 INFO:teuthology.orchestra.run.smithi120.stdout:849 read 0xea93ec thru 0xeb4259 (0xae6e bytes) 2024-09-18T01:08:08.070 INFO:teuthology.orchestra.run.smithi120.stdout:850 punch from 0x20706ef to 0x2078417, (0x7d28 bytes) 2024-09-18T01:08:08.070 INFO:teuthology.orchestra.run.smithi120.stdout:851 read 0x179487d thru 0x179e20a (0x998e bytes) 2024-09-18T01:08:08.070 INFO:teuthology.orchestra.run.smithi120.stdout:852 write 0x43b2ffa thru 0x43bd6dd (0xa6e4 bytes) 2024-09-18T01:08:08.072 INFO:teuthology.orchestra.run.smithi120.stdout:853 punch from 0x7d52fac to 0x7d57306, (0x435a bytes) 2024-09-18T01:08:08.073 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.073 INFO:teuthology.orchestra.run.smithi120.stdout:855 read 0x39310d7 thru 0x3937ade (0x6a08 bytes) 2024-09-18T01:08:08.073 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.073 INFO:teuthology.orchestra.run.smithi120.stdout:859 writesame 0x25e615e thru 0x25f0d61 data_size 0x4(0xac04 bytes) 2024-09-18T01:08:08.086 INFO:teuthology.orchestra.run.smithi120.stdout:861 trunc from 0xd81b1f7 to 0x9c5912a 2024-09-18T01:08:08.095 INFO:teuthology.orchestra.run.smithi120.stdout:862 read 0x6f05686 thru 0x6f0689e (0x1219 bytes) 2024-09-18T01:08:08.102 INFO:teuthology.orchestra.run.smithi120.stdout:863 trunc from 0x9c5912a to 0x4074c75 2024-09-18T01:08:08.114 INFO:teuthology.orchestra.run.smithi120.stdout:864 read 0x37a538d thru 0x37a9623 (0x4297 bytes) 2024-09-18T01:08:08.114 INFO:teuthology.orchestra.run.smithi120.stdout:865 read 0x186d5c5 thru 0x1871c4a (0x4686 bytes) 2024-09-18T01:08:08.132 INFO:teuthology.orchestra.run.smithi120.stdout:866 write 0xb6a3131 thru 0xb6b0a0c (0xd8dc bytes) 2024-09-18T01:08:08.134 INFO:teuthology.orchestra.run.smithi120.stdout:867 read 0x8e91f08 thru 0x8e95e9c (0x3f95 bytes) 2024-09-18T01:08:08.135 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.135 INFO:teuthology.orchestra.run.smithi120.stdout:869 write 0x2076fd2 thru 0x2086fa5 (0xffd4 bytes) 2024-09-18T01:08:08.137 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:08.137 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:08.137 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:08.137 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:08.137 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:08.139 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:08.139 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:08.145 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:08.144+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:08.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:08.152 INFO:teuthology.orchestra.run.smithi120.stdout:870 trunc from 0xb6b0a0d to 0x7ef0f0 2024-09-18T01:08:08.170 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:08.176 INFO:teuthology.orchestra.run.smithi120.stdout:871 write 0x1e4f0a3 thru 0x1e580e0 (0x903e bytes) 2024-09-18T01:08:08.180 INFO:teuthology.orchestra.run.smithi120.stdout:872 punch from 0x157ef00 to 0x15857ea, (0x68ea bytes) 2024-09-18T01:08:08.180 INFO:teuthology.orchestra.run.smithi120.stdout:873 read 0x15f8802 thru 0x1602872 (0xa071 bytes) 2024-09-18T01:08:08.181 INFO:teuthology.orchestra.run.smithi120.stdout:874 read 0x2f2705 thru 0x2f9f7b (0x7877 bytes) 2024-09-18T01:08:08.182 INFO:teuthology.orchestra.run.smithi120.stdout:875 read 0x108b4d9 thru 0x1099034 (0xdb5c bytes) 2024-09-18T01:08:08.190 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:08.190 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:08.196 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:08:08.196 INFO:teuthology.orchestra.run.smithi120.stdout:876 trunc from 0x1e580e1 to 0xc80e050 2024-09-18T01:08:08.201 INFO:teuthology.orchestra.run.smithi120.stdout:878 write 0xc09ce3b thru 0xc0ab06d (0xe233 bytes) 2024-09-18T01:08:08.205 INFO:teuthology.orchestra.run.smithi120.stdout:879 write 0x6272f8a thru 0x62812ea (0xe361 bytes) 2024-09-18T01:08:08.210 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:08.210 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:08.210 INFO:teuthology.orchestra.run.smithi120.stdout:881 write 0xa8076ee thru 0xa80e083 (0x6996 bytes) 2024-09-18T01:08:08.213 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.213 INFO:teuthology.orchestra.run.smithi120.stdout:885 write 0x6d5ccd1 thru 0x6d62f1e (0x624e bytes) 2024-09-18T01:08:08.227 INFO:teuthology.orchestra.run.smithi120.stdout:886 trunc from 0xc80e050 to 0x275c74a 2024-09-18T01:08:08.228 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:08.228 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:08.245 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:08.244+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:08.248 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:08.248 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:08.249 INFO:teuthology.orchestra.run.smithi120.stdout:887 trunc from 0x275c74a to 0x86c4471 2024-09-18T01:08:08.267 INFO:teuthology.orchestra.run.smithi120.stdout:888 write 0xbeeb9eb thru 0xbef3f82 (0x8598 bytes) 2024-09-18T01:08:08.268 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:08.283 INFO:teuthology.orchestra.run.smithi120.stdout:889 trunc from 0xbef3f83 to 0x1826d32 2024-09-18T01:08:08.289 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:08.297 INFO:teuthology.orchestra.run.smithi120.stdout:890 punch from 0xf4d2ea to 0xf5b3bf, (0xe0d5 bytes) 2024-09-18T01:08:08.298 INFO:teuthology.orchestra.run.smithi120.stdout:891 trunc from 0x1826d32 to 0x26bd0bf 2024-09-18T01:08:08.300 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:08.300 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:08.304 INFO:teuthology.orchestra.run.smithi120.stdout:892 read 0x1565c38 thru 0x15745fa (0xe9c3 bytes) 2024-09-18T01:08:08.305 INFO:teuthology.orchestra.run.smithi120.stdout:893 read 0x461ae3 thru 0x46448b (0x29a9 bytes) 2024-09-18T01:08:08.305 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.305 INFO:teuthology.orchestra.run.smithi120.stdout:896 read 0x1f71c26 thru 0x1f7cc4d (0xb028 bytes) 2024-09-18T01:08:08.312 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:08.318 INFO:teuthology.orchestra.run.smithi120.stdout:897 trunc from 0x26bd0bf to 0xc956e35 2024-09-18T01:08:08.324 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.325 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.325 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.340 INFO:teuthology.orchestra.run.smithi120.stdout:904 trunc from 0xc956e35 to 0x5cc16c 2024-09-18T01:08:08.346 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:08.345+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:08.364 INFO:teuthology.orchestra.run.smithi120.stdout:905 write 0x48db4a9 thru 0x48e7653 (0xc1ab bytes) 2024-09-18T01:08:08.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:08:08.377 INFO:teuthology.orchestra.run.smithi120.stdout:906 writesame 0x73fb726 thru 0x7409384 data_size 0x5(0xdc5f bytes) 2024-09-18T01:08:08.379 INFO:teuthology.orchestra.run.smithi120.stdout:907 read 0x29eddb7 thru 0x29f87dd (0xaa27 bytes) 2024-09-18T01:08:08.394 INFO:teuthology.orchestra.run.smithi120.stdout:908 writesame 0xeb47ba1 thru 0xeb53cd2 data_size 0x3(0xc132 bytes) 2024-09-18T01:08:08.394 INFO:teuthology.orchestra.run.smithi120.stdout:910 write 0xcc859a3 thru 0xcc89b3d (0x419b bytes) 2024-09-18T01:08:08.406 INFO:teuthology.orchestra.run.smithi120.stdout:913 trunc from 0xeb53cd3 to 0x7a2a379 2024-09-18T01:08:08.427 INFO:teuthology.orchestra.run.smithi120.stdout:914 write 0x750647c thru 0x750d966 (0x74eb bytes) 2024-09-18T01:08:08.430 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.440 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:08.440 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:08.441 INFO:teuthology.orchestra.run.smithi120.stdout:917 write 0xbf36a22 thru 0xbf439d9 (0xcfb8 bytes) 2024-09-18T01:08:08.445 INFO:teuthology.orchestra.run.smithi120.stdout:919 write 0x463c004 thru 0x4646a89 (0xaa86 bytes) 2024-09-18T01:08:08.447 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:08.446+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:08.448 INFO:teuthology.orchestra.run.smithi120.stdout:921 writesame 0x545bec thru 0x54a6ce data_size 0x1(0x4ae3 bytes) 2024-09-18T01:08:08.449 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:922 writesame 0x85181 thru 0x87b02 data_size 0x17(0x2982 bytes) 2024-09-18T01:08:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:924 read 0x883ecdd thru 0x883fee1 (0x1205 bytes) 2024-09-18T01:08:08.468 INFO:teuthology.orchestra.run.smithi120.stdout:925 write 0xe25a06e thru 0xe2662ec (0xc27f bytes) 2024-09-18T01:08:08.472 INFO:teuthology.orchestra.run.smithi120.stdout:926 read 0xce250db thru 0xce27791 (0x26b7 bytes) 2024-09-18T01:08:08.488 INFO:teuthology.orchestra.run.smithi120.stdout:927 trunc from 0xe2662ed to 0x29064a1 2024-09-18T01:08:08.514 INFO:teuthology.orchestra.run.smithi120.stdout:929 write 0x84e41b7 thru 0x84e5270 (0x10ba bytes) 2024-09-18T01:08:08.514 INFO:teuthology.orchestra.run.smithi120.stdout:930 write 0x631fa35 thru 0x632866e (0x8c3a bytes) 2024-09-18T01:08:08.516 INFO:teuthology.orchestra.run.smithi120.stdout:931 punch from 0x1186abb to 0x119528c, (0xe7d1 bytes) 2024-09-18T01:08:08.516 INFO:teuthology.orchestra.run.smithi120.stdout:932 read 0x475ef56 thru 0x47647aa (0x5855 bytes) 2024-09-18T01:08:08.516 INFO:teuthology.orchestra.run.smithi120.stdout:933 writesame 0x308953a thru 0x308a88d data_size 0x2(0x1354 bytes) 2024-09-18T01:08:08.531 INFO:teuthology.orchestra.run.smithi120.stdout:934 write 0xe425493 thru 0xe427dab (0x2919 bytes) 2024-09-18T01:08:08.534 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.534 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.534 INFO:teuthology.orchestra.run.smithi120.stdout:937 write 0x564adc9 thru 0x56525aa (0x77e2 bytes) 2024-09-18T01:08:08.536 INFO:teuthology.orchestra.run.smithi120.stdout:938 writesame 0x7ac7a14 thru 0x7aca916 data_size 0x1d(0x2f03 bytes) 2024-09-18T01:08:08.537 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.537 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.537 INFO:teuthology.orchestra.run.smithi120.stdout:941 read 0x36df04e thru 0x36e4272 (0x5225 bytes) 2024-09-18T01:08:08.543 INFO:teuthology.orchestra.run.smithi120.stdout:942 write 0xe4876fa thru 0xe49387c (0xc183 bytes) 2024-09-18T01:08:08.545 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.545 INFO:teuthology.orchestra.run.smithi120.stdout:944 read 0xf0af29 thru 0xf11703 (0x67db bytes) 2024-09-18T01:08:08.546 INFO:teuthology.orchestra.run.smithi120.stdout:945 write 0xcc882e1 thru 0xcc96a61 (0xe781 bytes) 2024-09-18T01:08:08.547 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:08.547+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:08.549 INFO:teuthology.orchestra.run.smithi120.stdout:946 read 0xded7b81 thru 0xdedd9e3 (0x5e63 bytes) 2024-09-18T01:08:08.549 INFO:teuthology.orchestra.run.smithi120.stdout:949 read 0x6a54af7 thru 0x6a62496 (0xd9a0 bytes) 2024-09-18T01:08:08.550 INFO:teuthology.orchestra.run.smithi120.stdout:950 punch from 0xb2f1abf to 0xb3005db, (0xeb1c bytes) 2024-09-18T01:08:08.550 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.550 INFO:teuthology.orchestra.run.smithi120.stdout:952 punch from 0x3878c74 to 0x3880d26, (0x80b2 bytes) 2024-09-18T01:08:08.550 INFO:teuthology.orchestra.run.smithi120.stdout:953 write 0x2e1fd52 thru 0x2e27112 (0x73c1 bytes) 2024-09-18T01:08:08.551 INFO:teuthology.orchestra.run.smithi120.stdout:954 writesame 0xa9c638b thru 0xa9cb68d data_size 0x1(0x5303 bytes) 2024-09-18T01:08:08.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:08:08.560 INFO:teuthology.orchestra.run.smithi120.stdout:956 writesame 0xaedcfa1 thru 0xaedef61 data_size 0x1(0x1fc1 bytes) 2024-09-18T01:08:08.566 INFO:teuthology.orchestra.run.smithi120.stdout:957 writesame 0x5e9fb93 thru 0x5ea9723 data_size 0x4b(0x9b91 bytes) 2024-09-18T01:08:08.568 INFO:teuthology.orchestra.run.smithi120.stdout:959 writesame 0x86a3895 thru 0x86ab4d0 data_size 0x2(0x7c3c bytes) 2024-09-18T01:08:08.576 INFO:teuthology.orchestra.run.smithi120.stdout:960 writesame 0xa3faadb thru 0xa3fc661 data_size 0x1d(0x1b87 bytes) 2024-09-18T01:08:08.586 INFO:teuthology.orchestra.run.smithi120.stdout:961 trunc from 0xe49387d to 0x8a081c1 2024-09-18T01:08:08.590 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:08.590 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:08.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:08.602 INFO:teuthology.orchestra.run.smithi120.stdout:962 read 0x5cb5a41 thru 0x5cb7161 (0x1721 bytes) 2024-09-18T01:08:08.602 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.602 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.614 INFO:teuthology.orchestra.run.smithi120.stdout:968 writesame 0xc70219e thru 0xc7026b2 data_size 0x1(0x515 bytes) 2024-09-18T01:08:08.614 INFO:teuthology.orchestra.run.smithi120.stdout:969 read 0x38b49dd thru 0x38b595a (0xf7e bytes) 2024-09-18T01:08:08.622 INFO:teuthology.orchestra.run.smithi120.stdout:972 write 0xd9c3fef thru 0xd9c5927 (0x1939 bytes) 2024-09-18T01:08:08.625 INFO:teuthology.orchestra.run.smithi120.stdout:973 punch from 0xa89bece to 0xa8a000a, (0x413c bytes) 2024-09-18T01:08:08.625 INFO:teuthology.orchestra.run.smithi120.stdout:974 punch from 0x5e0487f to 0x5e124f8, (0xdc79 bytes) 2024-09-18T01:08:08.626 INFO:teuthology.orchestra.run.smithi120.stdout:977 read 0x738ef8c thru 0x73969f9 (0x7a6e bytes) 2024-09-18T01:08:08.626 INFO:teuthology.orchestra.run.smithi120.stdout:979 read 0x6737c34 thru 0x6746d31 (0xf0fe bytes) 2024-09-18T01:08:08.626 INFO:teuthology.orchestra.run.smithi120.stdout:980 punch from 0x7e516c5 to 0x7e599b4, (0x82ef bytes) 2024-09-18T01:08:08.639 INFO:teuthology.orchestra.run.smithi120.stdout:981 trunc from 0xd9c5928 to 0x2bbb4da 2024-09-18T01:08:08.648 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:08.647+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:08.669 INFO:teuthology.orchestra.run.smithi120.stdout:982 write 0xc8812dd thru 0xc882f51 (0x1c75 bytes) 2024-09-18T01:08:08.672 INFO:teuthology.orchestra.run.smithi120.stdout:983 read 0xbd3fc17 thru 0xbd43aaa (0x3e94 bytes) 2024-09-18T01:08:08.672 INFO:teuthology.orchestra.run.smithi120.stdout:984 punch from 0x231a355 to 0x23275fb, (0xd2a6 bytes) 2024-09-18T01:08:08.672 INFO:teuthology.orchestra.run.smithi120.stdout:985 read 0x9499511 thru 0x949e1a4 (0x4c94 bytes) 2024-09-18T01:08:08.672 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:08.673 INFO:teuthology.orchestra.run.smithi120.stdout:987 writesame 0x3ff1f83 thru 0x3ffa7d5 data_size 0x1(0x8853 bytes) 2024-09-18T01:08:08.688 INFO:teuthology.orchestra.run.smithi120.stdout:988 read 0xa37f226 thru 0xa384a4e (0x5829 bytes) 2024-09-18T01:08:08.689 INFO:teuthology.orchestra.run.smithi120.stdout:990 read 0x3cd5c7 thru 0x3dbf49 (0xe983 bytes) 2024-09-18T01:08:08.689 INFO:teuthology.orchestra.run.smithi120.stdout:992 clone 12 order 18 su 16384 sc 10 2024-09-18T01:08:08.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:08:08.719 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:08.719 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:08.728 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:08.749 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:08.749+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:08.751 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:08:08.850 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:08.849+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:08.874 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T01:08:08.878 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:08.878 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:08.887 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:08.951 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:08.949+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:09.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T01:08:09.036 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.037 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:09.051 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:09.050+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:09.153 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:09.151+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:09.155 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T01:08:09.192 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.192 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.201 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:09.253 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:09.251+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:09.306 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.307 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:09.356 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:09.354+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:09.455 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:09.453+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:09.458 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.458 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:09.542 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:08:09.555 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:09.555+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:09.616 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.616 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.625 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:09.669 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:09.667+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:09.728 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.729 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:09.780 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:09.777+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:09.859 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:09.858+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:09.882 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.882 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:09.891 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:09.960 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:09.958+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:09.995 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:09.995 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:10.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:10.061 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:10.060+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:10.147 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:10.148 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:10.157 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:10.162 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:10.161+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:10.261 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:10.261 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:10.262 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:10.260+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:10.269 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:10.306 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:10.362 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:10.360+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:10.413 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:10.414 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:10.422 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:10.439 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:10.440 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:10.456 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:10.457 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:10.463 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:10.461+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:10.472 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:10.473 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:10.488 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:10.488 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:10.504 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:10.505 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:10.521 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:10.521 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:10.537 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:10.538 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:10.557 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:10.557 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:10.564 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:10.562+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:10.569 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:10.569 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:10.569 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:10.569 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:10.578 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:10.666 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:10.664+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:10.750 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:10.751 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:10.751 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:10.751 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:10.751 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:10.761 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:10.802 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:10.800+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:10.867 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:10.865+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.035329412000003, 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.642150114000003, 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.275418813999998, 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12826 3.d 3:b63623b7:::rbd_data.1176ec0d676b.0000000000000029:head [sparse-read 0~1044659] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.186266944, 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014765014999999999, 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12826 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746871+0000", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746872+0000", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746874+0000", 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746884+0000", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0088999999999999e-05 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.746889+0000", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.259e-06 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758702+0000", 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011813259 2024-09-18T01:08:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758789+0000", 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6386999999999998e-05 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761636+0000", 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002847488 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.996031665, 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:10.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:10.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:10.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.303709941000001, 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:10.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.303694843999999, 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:10.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.975 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.303683694, 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:10.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.303674056999998, 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:10.980 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:10.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.299762008999998, 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:10.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.989 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17718 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.298728753999999, 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015103220000000001, 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17718 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634409+0000", 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634413+0000", 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634417+0000", 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5689999999999998e-06 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634429+0000", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2082000000000001e-05 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.634445+0000", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5818e-05 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649397+0000", 2024-09-18T01:08:10.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014951391 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649457+0000", 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0001999999999999e-05 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649513+0000", 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066999999999997e-05 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17734 3.12 e46/23 3:4cb2c02c:::rbd_data.11762d95e74a.0000000000000002:head v 46'79, mlcod=44'77)", 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.296798857999999, 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014297084999999999, 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636340+0000", 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:08:10.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636751+0000", 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041088000000000002 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636760+0000", 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1640000000000007e-06 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636767+0000", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0219999999999999e-06 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636823+0000", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6632000000000003e-05 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636844+0000", 2024-09-18T01:08:10.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.056e-05 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650572+0000", 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013728290000000001 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650636+0000", 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4144000000000001e-05 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:10.999 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.2362607590000003, 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:11.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:11.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:38007 3.12 e60/23 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:head v 60'103, mlcod=51'101)", 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.2291351190000004, 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015053626000000001, 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.704003+0000", 2024-09-18T01:08:11.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6399999999999998e-07 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711136+0000", 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071328030000000001 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711149+0000", 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3047000000000001e-05 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711157+0000", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711210+0000", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2818999999999997e-05 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711227+0000", 2024-09-18T01:08:11.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7688000000000001e-05 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.719021+0000", 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077937309999999999 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.719057+0000", 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5644e-05 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39021 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 212992~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645763+0000", 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28737511599999999, 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013307536999999999, 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39021 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645763+0000", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645763+0000", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645763+0000", 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1899999999999999e-07 2024-09-18T01:08:11.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645764+0000", 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.271e-06 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645768+0000", 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1250000000000001e-06 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645770+0000", 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.514e-06 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.658845+0000", 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013074453999999999 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.658888+0000", 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3603999999999998e-05 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659071+0000", 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018234999999999999 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39022 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645773+0000", 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.287364972, 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013598298, 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39022 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645773+0000", 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645773+0000", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645773+0000", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645775+0000", 2024-09-18T01:08:11.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-06 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645778+0000", 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645780+0000", 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4480000000000001e-06 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659109+0000", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013329135000000001 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659158+0000", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8607999999999999e-05 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659371+0000", 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021357799999999999 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.019 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39023 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 540672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645783+0000", 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28735515099999998, 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013831533, 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39023 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645783+0000", 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645783+0000", 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645783+0000", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645784+0000", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1200000000000001e-06 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645787+0000", 2024-09-18T01:08:11.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0730000000000001e-06 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645790+0000", 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659389+0000", 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013598837000000001 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659433+0000", 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4520000000000001e-05 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659614+0000", 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018123600000000001 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39024 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.024 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645794+0000", 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28734456200000003, 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014165247000000001, 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39024 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645794+0000", 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645794+0000", 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645794+0000", 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1899999999999999e-07 2024-09-18T01:08:11.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645795+0000", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0699999999999999e-06 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645798+0000", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645800+0000", 2024-09-18T01:08:11.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659648+0000", 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013847976999999999 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659704+0000", 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5213999999999997e-05 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659959+0000", 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025525100000000001 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39025 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 868352~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645803+0000", 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28733492199999999, 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014410360000000001, 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.029 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39025 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645803+0000", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645803+0000", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645803+0000", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645804+0000", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-07 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645808+0000", 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645811+0000", 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.605e-06 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.659974+0000", 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014162816 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.660017+0000", 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3064e-05 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.660214+0000", 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000196839 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39026 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.645816+0000", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28732166799999997, 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014678508999999999, 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39026 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:11.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645816+0000", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645816+0000", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645817+0000", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6300000000000001e-07 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:11.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645821+0000", 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.352e-06 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645824+0000", 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3069999999999998e-06 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.645827+0000", 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000002e-06 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.660232+0000", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014405256 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.660277+0000", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4610000000000001e-05 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.660495+0000", 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000218261 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.038 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:11.038 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:11.038 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:11.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:11.040 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:11.038+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:11.140 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:11.138+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:43465 3.18 e63/23 3:1e554e44:::rbd_data.11762bc2f01c.0000000000000291:head v 63'88, mlcod=63'86)", 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:11.182556+0000", 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.00068654499999999999, 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00070704600000000002, 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182556+0000", 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182556+0000", 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182557+0000", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.09e-07 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182638+0000", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0677e-05 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182652+0000", 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4096e-05 2024-09-18T01:08:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182660+0000", 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8169999999999999e-06 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182710+0000", 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0288999999999999e-05 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:11.182737+0000", 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6772e-05 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-09-18T01:08:11.188 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:11.196 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:11.240 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:11.238+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:11.298 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:11.298 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:11.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:11.341 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:11.340+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:11.423 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.834293407000001, 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.830659654000002, 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.830636085999998, 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.829060684000002, 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.824627592999999, 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.823305416, 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.819251605000002, 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.793563175999999, 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.763024971, 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.762535279000002, 2024-09-18T01:08:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.698175582999999, 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52713795100000005, 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52710918500000004, 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52708593800000003, 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52706593499999999, 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.527047292, 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52702875000000005, 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52700816399999995, 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.52696389399999999, 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.525983702, 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.530 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:11.532 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:11.532 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:11.530+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:11.560 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:08:11.632 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:11.631+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:11.637 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:11.637 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:11.647 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:11.733 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:11.732+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:11.800 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:11.801 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:11.810 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:11.834 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:11.833+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:11.856 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-18T01:08:11.927 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747142560999997, 2024-09-18T01:08:11.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:11.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747123756999997, 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:11.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:11.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:11.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747107188999998, 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747091003000001, 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747079434, 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747070067999999, 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747057884999997, 2024-09-18T01:08:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747042190000002, 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747025035999997, 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.747002870000003, 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746986419999999, 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746976965999998, 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746942914999998, 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746923191, 2024-09-18T01:08:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746904954000001, 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746884379000001, 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746865151999998, 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746845798999999, 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.034 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.746827144000001, 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:12.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:12.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:12.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.738045194999998, 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:12.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.049 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:12.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:12.052 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:12.049+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:12.075 INFO:teuthology.orchestra.run.smithi120.stdout:993 read 0x3aef219 thru 0x3af8f21 (0x9d09 bytes) 2024-09-18T01:08:12.076 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.076 INFO:teuthology.orchestra.run.smithi120.stdout:995 write 0x9e43308 thru 0x9e4b6b7 (0x83b0 bytes) 2024-09-18T01:08:12.091 INFO:teuthology.orchestra.run.smithi120.stdout:997 read 0x85a9f59 thru 0x85afcb6 (0x5d5e bytes) 2024-09-18T01:08:12.091 INFO:teuthology.orchestra.run.smithi120.stdout:998 write 0x90be465 thru 0x90c5b3b (0x76d7 bytes) 2024-09-18T01:08:12.106 INFO:teuthology.orchestra.run.smithi120.stdout:1000 writesame 0xe46d7b5 thru 0xe474f10 data_size 0x2(0x775c bytes) 2024-09-18T01:08:12.106 INFO:teuthology.orchestra.run.smithi120.stdout:1001 write 0x239cacb thru 0x23a7ee5 (0xb41b bytes) 2024-09-18T01:08:12.115 INFO:teuthology.orchestra.run.smithi120.stdout:1003 trunc from 0xe474f11 to 0x843977f 2024-09-18T01:08:12.146 INFO:teuthology.orchestra.run.smithi120.stdout:1004 read 0x3a7f319 thru 0x3a87bec (0x88d4 bytes) 2024-09-18T01:08:12.146 INFO:teuthology.orchestra.run.smithi120.stdout:1005 writesame 0x55e4693 thru 0x55e7773 data_size 0x2b(0x30e1 bytes) 2024-09-18T01:08:12.150 INFO:teuthology.orchestra.run.smithi120.stdout:1007 read 0x14817d3 thru 0x1488ec6 (0x76f4 bytes) 2024-09-18T01:08:12.151 INFO:teuthology.orchestra.run.smithi120.stdout:1009 read 0x4fa0b4c thru 0x4fa331d (0x27d2 bytes) 2024-09-18T01:08:12.151 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:12.149+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:12.154 INFO:teuthology.orchestra.run.smithi120.stdout:1010 trunc from 0x843977f to 0x547e41c 2024-09-18T01:08:12.167 INFO:teuthology.orchestra.run.smithi120.stdout:1011 read 0x6eccf3 thru 0x6f9d2c (0xd03a bytes) 2024-09-18T01:08:12.168 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:12.168 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:12.168 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:12.168 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:12.168 INFO:teuthology.orchestra.run.smithi120.stdout:1012 read 0x25de098 thru 0x25e95bb (0xb524 bytes) 2024-09-18T01:08:12.180 INFO:teuthology.orchestra.run.smithi120.stdout:1013 writesame 0x9d3d736 thru 0x9d49608 data_size 0x1(0xbed3 bytes) 2024-09-18T01:08:12.180 INFO:teuthology.orchestra.run.smithi120.stdout:1014 write 0x17793f5 thru 0x17887cb (0xf3d7 bytes) 2024-09-18T01:08:12.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:12.182 INFO:teuthology.orchestra.run.smithi120.stdout:1015 write 0x1977db thru 0x19d177 (0x599d bytes) 2024-09-18T01:08:12.187 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.187 INFO:teuthology.orchestra.run.smithi120.stdout:1017 write 0x95694fa thru 0x956d269 (0x3d70 bytes) 2024-09-18T01:08:12.192 INFO:teuthology.orchestra.run.smithi120.stdout:1018 punch from 0x1ec276b to 0x1ec82b9, (0x5b4e bytes) 2024-09-18T01:08:12.192 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.196 INFO:teuthology.orchestra.run.smithi120.stdout:1022 trunc from 0x9d49609 to 0x63f2902 2024-09-18T01:08:12.205 INFO:teuthology.orchestra.run.smithi120.stdout:1023 read 0x213b182 thru 0x213e3c7 (0x3246 bytes) 2024-09-18T01:08:12.206 INFO:teuthology.orchestra.run.smithi120.stdout:1024 read 0x30adb31 thru 0x30bcfe3 (0xf4b3 bytes) 2024-09-18T01:08:12.206 INFO:teuthology.orchestra.run.smithi120.stdout:1025 read 0x3c0e753 thru 0x3c11375 (0x2c23 bytes) 2024-09-18T01:08:12.206 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.218 INFO:teuthology.orchestra.run.smithi120.stdout:1027 write 0xb4bb04f thru 0xb4c5d77 (0xad29 bytes) 2024-09-18T01:08:12.223 INFO:teuthology.orchestra.run.smithi120.stdout:1028 punch from 0x1f68ab4 to 0x1f69c57, (0x11a3 bytes) 2024-09-18T01:08:12.223 INFO:teuthology.orchestra.run.smithi120.stdout:1031 read 0xae7a60f thru 0xae8371e (0x9110 bytes) 2024-09-18T01:08:12.224 INFO:teuthology.orchestra.run.smithi120.stdout:1032 write 0x237077f thru 0x2380414 (0xfc96 bytes) 2024-09-18T01:08:12.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.228 INFO:teuthology.orchestra.run.smithi120.stdout:1038 writesame 0xa9aa2a8 thru 0xa9aeb8c data_size 0x1(0x48e5 bytes) 2024-09-18T01:08:12.235 INFO:teuthology.orchestra.run.smithi120.stdout:1039 writesame 0x4c33f94 thru 0x4c40b22 data_size 0x1f(0xcb8f bytes) 2024-09-18T01:08:12.244 INFO:teuthology.orchestra.run.smithi120.stdout:1040 write 0x1e3e81d thru 0x1e49243 (0xaa27 bytes) 2024-09-18T01:08:12.252 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:12.250+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:12.260 INFO:teuthology.orchestra.run.smithi120.stdout:1042 trunc from 0xb4c5d78 to 0x2af5f89 2024-09-18T01:08:12.290 INFO:teuthology.orchestra.run.smithi120.stdout:1044 trunc from 0x2af5f89 to 0x67d7467 2024-09-18T01:08:12.296 INFO:teuthology.orchestra.run.smithi120.stdout:1045 read 0x4b5a149 thru 0x4b5c6a5 (0x255d bytes) 2024-09-18T01:08:12.296 INFO:teuthology.orchestra.run.smithi120.stdout:1046 writesame 0x4244875 thru 0x4248a12 data_size 0x2(0x419e bytes) 2024-09-18T01:08:12.300 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:12.300 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:12.303 INFO:teuthology.orchestra.run.smithi120.stdout:1047 read 0x1505483 thru 0x150b52d (0x60ab bytes) 2024-09-18T01:08:12.304 INFO:teuthology.orchestra.run.smithi120.stdout:1050 read 0xbec000 thru 0xbf8c8a (0xcc8b bytes) 2024-09-18T01:08:12.305 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.305 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.306 INFO:teuthology.orchestra.run.smithi120.stdout:1053 punch from 0x4c71039 to 0x4c7834a, (0x7311 bytes) 2024-09-18T01:08:12.306 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.310 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:12.319 INFO:teuthology.orchestra.run.smithi120.stdout:1055 write 0xd973261 thru 0xd97cbec (0x998c bytes) 2024-09-18T01:08:12.326 INFO:teuthology.orchestra.run.smithi120.stdout:1056 write 0x6d388ae thru 0x6d41040 (0x8793 bytes) 2024-09-18T01:08:12.330 INFO:teuthology.orchestra.run.smithi120.stdout:1057 punch from 0x6944051 to 0x694bc38, (0x7be7 bytes) 2024-09-18T01:08:12.330 INFO:teuthology.orchestra.run.smithi120.stdout:1058 writesame 0x6e067e2 thru 0x6e0b596 data_size 0x13(0x4db5 bytes) 2024-09-18T01:08:12.334 INFO:teuthology.orchestra.run.smithi120.stdout:1059 punch from 0x7829604 to 0x782ef7a, (0x5976 bytes) 2024-09-18T01:08:12.334 INFO:teuthology.orchestra.run.smithi120.stdout:1061 writesame 0x7d14650 thru 0x7d18e42 data_size 0x1(0x47f3 bytes) 2024-09-18T01:08:12.343 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.344 INFO:teuthology.orchestra.run.smithi120.stdout:1063 trunc from 0xd97cbed to 0xd31df98 2024-09-18T01:08:12.353 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:12.352+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:12.359 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.360 INFO:teuthology.orchestra.run.smithi120.stdout:1069 trunc from 0xd31df98 to 0xc871084 2024-09-18T01:08:12.365 INFO:teuthology.orchestra.run.smithi120.stdout:1070 writesame 0x8392eb9 thru 0x839dbfc data_size 0x1a(0xad44 bytes) 2024-09-18T01:08:12.377 INFO:teuthology.orchestra.run.smithi120.stdout:1071 read 0x131787c thru 0x131a5dc (0x2d61 bytes) 2024-09-18T01:08:12.386 INFO:teuthology.orchestra.run.smithi120.stdout:1072 writesame 0xe724151 thru 0xe72bda0 data_size 0x75(0x7c50 bytes) 2024-09-18T01:08:12.386 INFO:teuthology.orchestra.run.smithi120.stdout:1074 punch from 0x75e40cc to 0x75e785c, (0x3790 bytes) 2024-09-18T01:08:12.392 INFO:teuthology.orchestra.run.smithi120.stdout:1077 trunc from 0xe72bda1 to 0x9f78d9a 2024-09-18T01:08:12.397 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:12.397 INFO:teuthology.orchestra.run.smithi120.stdout:1079 clone 13 order 25 su 1048576 sc 3 2024-09-18T01:08:12.435 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.493930685000002, 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.366002645000002, 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.315628584999999, 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.225735368999999, 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.224139620999999, 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.223329002, 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.172781509000004, 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.839956190999999, 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.276356134, 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.258595048, 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.255938345000001, 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.673290802, 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.258977343000002, 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.757278321, 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.7557014340000001, 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.725953844, 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.6803106830000001, 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39974 3.4 e63/23 3:232e8047:::rbd_data.11762bc2f01c.000000000000003d:head v 63'122, mlcod=61'120)", 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.662820446, 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034513289999999999, 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770836+0000", 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3540000000000001e-06 2024-09-18T01:08:12.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770967+0000", 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130853 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770988+0000", 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0514e-05 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770998+0000", 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771055+0000", 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7376000000000001e-05 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771073+0000", 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8406999999999998e-05 2024-09-18T01:08:12.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774238+0000", 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003164287 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774286+0000", 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8838000000000002e-05 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.570574358, 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:12.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.566920928, 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:12.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:12.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:12.531 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:12.533 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:12.533 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:12.532+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:12.571 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:12.633 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:12.632+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:12.680 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:12.680 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:12.681 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:12.681 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:12.689 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:12.695 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:12.696 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:12.700 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:08:12.716 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:12.716 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:12.732 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:12.733 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:12.733 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:12.733+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:12.764 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:12.764 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:12.781 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:12.781 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:12.812 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:12.835 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:12.834+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:12.842 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:12.842 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:12.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:12.862 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:12.936 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:12.935+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:12.965 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:12.965 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:12.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:13.037 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:13.036+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:13.078 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.078 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:13.138 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:13.136+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:13.188 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.188 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:13.239 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:13.238+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:13.340 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:13.339+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:13.341 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.341 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.350 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:13.440 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:13.440+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:13.452 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.452 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.461 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:13.542 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:13.539+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:13.565 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.565 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.574 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:13.643 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:13.642+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:13.721 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.721 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.730 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:13.774 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:13.773+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:13.835 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.835 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.844 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:13.845 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:13.844+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:13.946 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:13.944+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:13.988 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:13.988 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:13.997 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:14.081 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:14.080+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:14.141 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.142 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.148 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:14.147+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:14.150 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:14.249 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:14.248+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:14.290 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:14.290 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:14.290 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-18T01:08:14.291 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:08:14.291 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:08:14.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-18T01:08:14.294 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.294 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.304 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:14.351 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:14.350+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:14.396 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:14.396 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:08:14.397 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:14.449 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.449 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.452 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:14.451+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:14.458 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:14.553 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:14.552+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:14.634 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.634 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:14.689 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:14.688+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:14.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:08:14.755 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:14.754+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:14.757 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.757 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.766 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:14.855 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:14.854+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:14.874 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:14.914 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:14.914 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:14.924 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:14.957 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:14.955+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:14.998 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:14.999 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:15.015 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:15.015 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:15.032 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:15.032 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:15.048 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:15.048 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:15.057 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:15.056+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:15.065 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:15.066 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.073 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.484405406, 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.074 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:15.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:15.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.480771652999998, 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.080 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:15.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.480748085000002, 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:15.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:15.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:15.090 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.479172683000002, 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:15.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:15.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:15.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.474739591999999, 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:15.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:15.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.104 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.473417415, 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:15.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:15.109 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.469363604000002, 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:15.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:15.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.443675175000003, 2024-09-18T01:08:15.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:15.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.413136969999996, 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:15.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:15.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:15.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:15.130 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.412647278000001, 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:15.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:15.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:15.135 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.348287582000001, 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:15.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:15.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1772499500000002, 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:15.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1772211840000004, 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.148 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:15.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1771979369999999, 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.153 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:15.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1771779340000004, 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:15.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1771592909999997, 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:15.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:15.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:15.167 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1771407490000003, 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:15.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:15.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.172 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1771201629999997, 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:15.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:15.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.177 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1770758929999996, 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:15.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1760957010000004, 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:15.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:15.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.187 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:15.190 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:15.190 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:15.190 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:15.190 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:15.191 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:15.191 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:15.192 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:15.188+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:15.290 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:15.289+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:15.330 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:15.331 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.339 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:15.391 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:15.390+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:15.442 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.442 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:15.442 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:15.442 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:15.442 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.451 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:15.492 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:15.490+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:15.593 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:15.592+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:15.596 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.596 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:15.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:15.596 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:15.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2177 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1409024~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416310211000003, 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023555162000000001, 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2177 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178200+0000", 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178203+0000", 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6249999999999999e-06 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178208+0000", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5090000000000001e-06 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178213+0000", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5029999999999999e-06 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201577+0000", 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023364116000000001 2024-09-18T01:08:15.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201654+0000", 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6889000000000002e-05 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201755+0000", 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010115600000000001 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416291407000003, 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:15.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416274839000003, 2024-09-18T01:08:15.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:15.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:15.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.613 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416258653, 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:15.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:15.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:15.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416247083999998, 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:15.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:15.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416237717999998, 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:15.623 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:15.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416225535000002, 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:15.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:15.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:15.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.41620984, 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:15.633 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:15.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:15.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416192686000002, 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:15.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416170520000001, 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:15.647 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416154069999997, 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:15.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416144615999997, 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:15.652 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416110564999997, 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.657 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:15.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416090840999999, 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.662 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:15.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:15.666 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416072604, 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:15.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:15.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416052028999999, 2024-09-18T01:08:15.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:15.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:15.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:15.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416032801999997, 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.676 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:15.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.680 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.416013448999998, 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:15.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.415994794, 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:15.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:15.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:15.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.407212844999997, 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:15.692 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:15.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:15.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:15.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:15.699 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:15.700 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:15.699+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:15.725 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:08:15.801 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:15.800+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:15.841 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.841 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:15.841 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:15.841 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.850 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-18T01:08:15.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:15.901 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:15.901+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:15.978 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:15.978 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:15.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:16.029 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:16.028+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:16.078 INFO:teuthology.orchestra.run.smithi120.stdout:1080 read 0x35c0d00 thru 0x35c0f3b (0x23c bytes) 2024-09-18T01:08:16.078 INFO:teuthology.orchestra.run.smithi120.stdout:1082 write 0x5ab9f8b thru 0x5ac4b04 (0xab7a bytes) 2024-09-18T01:08:16.084 INFO:teuthology.orchestra.run.smithi120.stdout:1084 read 0x55b1931 thru 0x55b9488 (0x7b58 bytes) 2024-09-18T01:08:16.092 INFO:teuthology.orchestra.run.smithi120.stdout:1085 writesame 0xba96c17 thru 0xba99381 data_size 0x1(0x276b bytes) 2024-09-18T01:08:16.099 INFO:teuthology.orchestra.run.smithi120.stdout:1086 trunc from 0xba99382 to 0xe2627f2 2024-09-18T01:08:16.103 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:16.102+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:16.108 INFO:teuthology.orchestra.run.smithi120.stdout:1087 read 0x217b4fb thru 0x2181c8d (0x6793 bytes) 2024-09-18T01:08:16.109 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.109 INFO:teuthology.orchestra.run.smithi120.stdout:1092 read 0xd525098 thru 0xd5283db (0x3344 bytes) 2024-09-18T01:08:16.109 INFO:teuthology.orchestra.run.smithi120.stdout:1095 write 0x9a1711f thru 0x9a24551 (0xd433 bytes) 2024-09-18T01:08:16.111 INFO:teuthology.orchestra.run.smithi120.stdout:1096 write 0x920fa1a thru 0x921a684 (0xac6b bytes) 2024-09-18T01:08:16.113 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.113 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.114 INFO:teuthology.orchestra.run.smithi120.stdout:1101 write 0xb46b995 thru 0xb471b8f (0x61fb bytes) 2024-09-18T01:08:16.116 INFO:teuthology.orchestra.run.smithi120.stdout:1102 punch from 0x5757bb6 to 0x5759d8c, (0x21d6 bytes) 2024-09-18T01:08:16.120 INFO:teuthology.orchestra.run.smithi120.stdout:1103 trunc from 0xe2627f2 to 0xd642da9 2024-09-18T01:08:16.124 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.124 INFO:teuthology.orchestra.run.smithi120.stdout:1108 read 0x347f3a8 thru 0x348047f (0x10d8 bytes) 2024-09-18T01:08:16.139 INFO:teuthology.orchestra.run.smithi120.stdout:1112 trunc from 0xd642da9 to 0x1aa1420 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.142 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.200326793000002, 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:16.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.072398753000002, 2024-09-18T01:08:16.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:16.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.022024692999999, 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.154 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:16.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:16.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:16.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:16.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.932131476999999, 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.160 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:16.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:16.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:16.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.930535728999999, 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:16.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:16.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:16.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.92972511, 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:16.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:16.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.879177617000003, 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:08:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:08:16.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.181 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.546352298999999, 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:16.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.982752242, 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:16.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:16.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.964991156, 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:16.190 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:16.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:16.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.962334453, 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.37968691, 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:16.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:16.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:16.203 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.965373451000001, 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:16.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:16.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:16.208 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.4636744290000001, 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:16.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.4620975420000004, 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.432349952, 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:16.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3867067909999999, 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39974 3.4 e63/23 3:232e8047:::rbd_data.11762bc2f01c.000000000000003d:head v 63'122, mlcod=61'120)", 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:16.227 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3692165540000003, 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034513289999999999, 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770836+0000", 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3540000000000001e-06 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770967+0000", 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130853 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770988+0000", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0514e-05 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770998+0000", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771055+0000", 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7376000000000001e-05 2024-09-18T01:08:16.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771073+0000", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8406999999999998e-05 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774238+0000", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003164287 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774286+0000", 2024-09-18T01:08:16.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8838000000000002e-05 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.2769704659999999, 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.232 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.236 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.2733170359999999, 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:16.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.241 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:16.243 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:1114 write 0x614960e thru 0x614a973 (0x1366 bytes) 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:1115 write 0xa1b260d thru 0xa1c00f4 (0xdae8 bytes) 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:1119 trunc from 0xa1c00f5 to 0x24b1b6a 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:1120 read 0xc520c6 thru 0xc5694a (0x4885 bytes) 2024-09-18T01:08:16.244 INFO:teuthology.orchestra.run.smithi120.stdout:1123 write 0xd9faa6e thru 0xda069da (0xbf6d bytes) 2024-09-18T01:08:16.245 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:16.242+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:16.245 INFO:teuthology.orchestra.run.smithi120.stdout:1124 write 0xb0ab43e thru 0xb0b5368 (0x9f2b bytes) 2024-09-18T01:08:16.247 INFO:teuthology.orchestra.run.smithi120.stdout:1125 write 0xa0bdfaa thru 0xa0ca513 (0xc56a bytes) 2024-09-18T01:08:16.250 INFO:teuthology.orchestra.run.smithi120.stdout:1126 writesame 0xd23ae52 thru 0xd23f476 data_size 0x1(0x4625 bytes) 2024-09-18T01:08:16.261 INFO:teuthology.orchestra.run.smithi120.stdout:1130 write 0x1adccaf thru 0x1ae94a9 (0xc7fb bytes) 2024-09-18T01:08:16.263 INFO:teuthology.orchestra.run.smithi120.stdout:1131 write 0x8df2b48 thru 0x8dfeb71 (0xc02a bytes) 2024-09-18T01:08:16.267 INFO:teuthology.orchestra.run.smithi120.stdout:1132 read 0x390fdbc thru 0x39113fb (0x1640 bytes) 2024-09-18T01:08:16.268 INFO:teuthology.orchestra.run.smithi120.stdout:1133 read 0x95cda39 thru 0x95d38e8 (0x5eb0 bytes) 2024-09-18T01:08:16.269 INFO:teuthology.orchestra.run.smithi120.stdout:1134 punch from 0x36b00b7 to 0x36b1e33, (0x1d7c bytes) 2024-09-18T01:08:16.269 INFO:teuthology.orchestra.run.smithi120.stdout:1135 punch from 0x38431e5 to 0x3846267, (0x3082 bytes) 2024-09-18T01:08:16.269 INFO:teuthology.orchestra.run.smithi120.stdout:1136 read 0xc66affb thru 0xc672d5f (0x7d65 bytes) 2024-09-18T01:08:16.279 INFO:teuthology.orchestra.run.smithi120.stdout:1137 trunc from 0xda069db to 0x5120bf5 2024-09-18T01:08:16.307 INFO:teuthology.orchestra.run.smithi120.stdout:1139 trunc from 0x5120bf5 to 0xe9ae99 2024-09-18T01:08:16.325 INFO:teuthology.orchestra.run.smithi120.stdout:1140 trunc from 0xe9ae99 to 0x44a7291 2024-09-18T01:08:16.334 INFO:teuthology.orchestra.run.smithi120.stdout:1141 writesame 0x13b0b8 thru 0x141d29 data_size 0x15(0x6c72 bytes) 2024-09-18T01:08:16.339 INFO:teuthology.orchestra.run.smithi120.stdout:1143 punch from 0x3a1dfa0 to 0x3a29718, (0xb778 bytes) 2024-09-18T01:08:16.339 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.339 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.339 INFO:teuthology.orchestra.run.smithi120.stdout:1147 read 0x3364e9a thru 0x336a3ec (0x5553 bytes) 2024-09-18T01:08:16.340 INFO:teuthology.orchestra.run.smithi120.stdout:1148 punch from 0x3d24b5a to 0x3d250a8, (0x54e bytes) 2024-09-18T01:08:16.340 INFO:teuthology.orchestra.run.smithi120.stdout:1149 punch from 0x411112d to 0x411c408, (0xb2db bytes) 2024-09-18T01:08:16.340 INFO:teuthology.orchestra.run.smithi120.stdout:1150 punch from 0x43b8f9f to 0x43c308c, (0xa0ed bytes) 2024-09-18T01:08:16.340 INFO:teuthology.orchestra.run.smithi120.stdout:1151 read 0x22cfd17 thru 0x22dd891 (0xdb7b bytes) 2024-09-18T01:08:16.344 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:16.343+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:16.352 INFO:teuthology.orchestra.run.smithi120.stdout:1154 write 0x97d0bef thru 0x97daaaf (0x9ec1 bytes) 2024-09-18T01:08:16.357 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:16.357 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:16.359 INFO:teuthology.orchestra.run.smithi120.stdout:1155 trunc from 0x97daab0 to 0xd859417 2024-09-18T01:08:16.365 INFO:teuthology.orchestra.run.smithi120.stdout:1156 writesame 0x80d248f thru 0x80d6516 data_size 0x46(0x4088 bytes) 2024-09-18T01:08:16.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:16.368 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.369 INFO:teuthology.orchestra.run.smithi120.stdout:1158 write 0x7f0f6a2 thru 0x7f19c87 (0xa5e6 bytes) 2024-09-18T01:08:16.370 INFO:teuthology.orchestra.run.smithi120.stdout:1159 writesame 0x9e81c32 thru 0x9e8e5d3 data_size 0x15(0xc9a2 bytes) 2024-09-18T01:08:16.372 INFO:teuthology.orchestra.run.smithi120.stdout:1160 write 0x8a3b29c thru 0x8a47f27 (0xcc8c bytes) 2024-09-18T01:08:16.376 INFO:teuthology.orchestra.run.smithi120.stdout:1163 writesame 0xb4a70be thru 0xb4aeb87 data_size 0x5d(0x7aca bytes) 2024-09-18T01:08:16.385 INFO:teuthology.orchestra.run.smithi120.stdout:1165 trunc from 0xd859417 to 0x8c3eaca 2024-09-18T01:08:16.394 INFO:teuthology.orchestra.run.smithi120.stdout:1166 read 0x6bc414d thru 0x6bc5e41 (0x1cf5 bytes) 2024-09-18T01:08:16.406 INFO:teuthology.orchestra.run.smithi120.stdout:1167 writesame 0xcbb493f thru 0xcbbd486 data_size 0x4(0x8b48 bytes) 2024-09-18T01:08:16.406 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:16.406 INFO:teuthology.orchestra.run.smithi120.stdout:1169 clone 14 order 20 su 524288 sc 7 2024-09-18T01:08:16.445 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:16.444+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:16.476 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:16.476 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:16.485 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:16.546 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:16.545+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38805 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2424832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0581012809999999, 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244350999999999, 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38805 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569048+0000", 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569057+0000", 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4530000000000008e-06 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569060+0000", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0170000000000001e-06 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611025+0000", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041964317000000001 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611073+0000", 2024-09-18T01:08:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8037000000000001e-05 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611288+0000", 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215596 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38806 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.058081456, 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042508381999999997, 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38806 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4400000000000001e-07 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569065+0000", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.206e-06 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569068+0000", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2890000000000002e-06 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569071+0000", 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:08:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611316+0000", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244918999999999 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611360+0000", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4239000000000001e-05 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611572+0000", 2024-09-18T01:08:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212186 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38807 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0580717030000004, 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042802420000000001, 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38807 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569074+0000", 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569075+0000", 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569078+0000", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9749999999999999e-06 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569080+0000", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611602+0000", 2024-09-18T01:08:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042521681999999998 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611650+0000", 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8179000000000001e-05 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611876+0000", 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022609299999999999 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38808 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.058061854, 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043110682999999997, 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38808 2024-09-18T01:08:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569084+0000", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569085+0000", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569088+0000", 2024-09-18T01:08:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569090+0000", 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3120000000000001e-06 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611901+0000", 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042810991999999999 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611955+0000", 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3983000000000001e-05 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612194+0000", 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239182 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38809 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 475136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0580435289999999, 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043389637000000002, 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38809 2024-09-18T01:08:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569122+0000", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0254999999999998e-05 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569127+0000", 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569130+0000", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.016e-06 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612224+0000", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043094406000000002 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612275+0000", 2024-09-18T01:08:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1054000000000003e-05 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612491+0000", 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215987 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38810 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.058010683, 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043652712000000003, 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38810 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569135+0000", 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569136+0000", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.083e-06 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569141+0000", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569143+0000", 2024-09-18T01:08:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3369999999999998e-06 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612521+0000", 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043377538 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612566+0000", 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5485000000000002e-05 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612787+0000", 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221158 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38811 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579994020000001, 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043930856999999997, 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38811 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569147+0000", 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0449999999999999e-06 2024-09-18T01:08:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569150+0000", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569152+0000", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612830+0000", 2024-09-18T01:08:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043677597999999998 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612880+0000", 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0219999999999997e-05 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613077+0000", 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019646600000000001 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38812 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579901930000002, 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044183260000000002, 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38812 2024-09-18T01:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569156+0000", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999991e-07 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569160+0000", 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3759999999999999e-06 2024-09-18T01:08:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569162+0000", 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613107+0000", 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043945166000000001 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613153+0000", 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613338+0000", 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018564800000000001 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38813 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1130496~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579807250000002, 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044519186000000002, 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38813 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569165+0000", 2024-09-18T01:08:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569166+0000", 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2399999999999996e-07 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569168+0000", 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9229999999999998e-06 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569178+0000", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6479999999999998e-06 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613370+0000", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044191800000000003 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613416+0000", 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6431999999999999e-05 2024-09-18T01:08:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613684+0000", 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026725300000000001 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38814 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579638940000002, 2024-09-18T01:08:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044848796000000003, 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38814 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569183+0000", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.096e-06 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569187+0000", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569191+0000", 2024-09-18T01:08:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1930000000000002e-06 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613735+0000", 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044543886999999997 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613784+0000", 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9858999999999998e-05 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614030+0000", 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024569700000000002 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38815 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1458176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.057951783, 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045101439, 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38815 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569194+0000", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569195+0000", 2024-09-18T01:08:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0389999999999999e-06 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569198+0000", 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9210000000000001e-06 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569200+0000", 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.367e-06 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614061+0000", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044861375000000002 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614104+0000", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3142000000000001e-05 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614295+0000", 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190383 2024-09-18T01:08:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38816 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1622016~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579426730000003, 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045464629999999999, 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38816 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569203+0000", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569206+0000", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.754e-06 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569211+0000", 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.109e-06 2024-09-18T01:08:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569213+0000", 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.379e-06 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614331+0000", 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045118395999999998 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614380+0000", 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614667+0000", 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028724000000000002 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38817 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1785856~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579292799999997, 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045799302, 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38817 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569217+0000", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.035e-06 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569221+0000", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.839e-06 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569223+0000", 2024-09-18T01:08:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3300000000000001e-06 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614711+0000", 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045487811000000003 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614772+0000", 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0729000000000001e-05 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615015+0000", 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024334399999999999 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38818 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579192300000004, 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046120525000000002, 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38818 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569230+0000", 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0260000000000001e-06 2024-09-18T01:08:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569234+0000", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569237+0000", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3750000000000001e-06 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615046+0000", 2024-09-18T01:08:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045809435000000003 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615096+0000", 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9654999999999998e-05 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615346+0000", 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025073700000000001 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38819 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2113536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0579034429999998, 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046434472999999997, 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38819 2024-09-18T01:08:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569243+0000", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.291e-06 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569247+0000", 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.031e-06 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569250+0000", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615395+0000", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046145046000000002 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615437+0000", 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2407999999999998e-05 2024-09-18T01:08:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615676+0000", 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239137 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38820 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2277376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0578925449999996, 2024-09-18T01:08:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046745223000000002, 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38820 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569254+0000", 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-07 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569258+0000", 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6489999999999999e-06 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569261+0000", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.322e-06 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615704+0000", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046442986999999998 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615764+0000", 2024-09-18T01:08:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9871000000000003e-05 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615998+0000", 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023420099999999999 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38821 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2441216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0578816929999997, 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047015754999999999, 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38821 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569264+0000", 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569265+0000", 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8599999999999996e-07 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569269+0000", 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8800000000000001e-06 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569271+0000", 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616017+0000", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046746593000000003 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616061+0000", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3894999999999999e-05 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616279+0000", 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000217875 2024-09-18T01:08:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38822 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0578716210000003, 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047344570000000002, 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38822 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569275+0000", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.026e-06 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569278+0000", 2024-09-18T01:08:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999999e-06 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569284+0000", 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.071e-06 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616318+0000", 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047033859999999997 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616374+0000", 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5636000000000001e-05 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616618+0000", 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024412699999999999 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38823 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0538950480000002, 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045935959999999998, 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38823 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573251+0000", 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573255+0000", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2520000000000001e-06 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573266+0000", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0958000000000001e-05 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573271+0000", 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1229999999999999e-06 2024-09-18T01:08:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616299+0000", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043027351999999998 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616351+0000", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2284000000000003e-05 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616681+0000", 2024-09-18T01:08:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033019400000000002 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619078+0000", 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023970749999999998 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619093+0000", 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4338999999999999e-05 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619143+0000", 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0346000000000002e-05 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619186+0000", 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3226999999999998e-05 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38824 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0485556989999996, 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042771339999999998, 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38824 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578590+0000", 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578594+0000", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0609999999999997e-06 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578605+0000", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0477e-05 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578611+0000", 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9540000000000001e-06 2024-09-18T01:08:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617084+0000", 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.038473680000000003 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617135+0000", 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0139000000000002e-05 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617316+0000", 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181082 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621302+0000", 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039860210000000002 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621309+0000", 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6490000000000008e-06 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621336+0000", 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6828000000000001e-05 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621361+0000", 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4621999999999999e-05 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:16.726 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:16.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:16.728 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:16.727+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:16.829 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:16.828+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:16.835 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:16.836 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:16.844 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:16.929 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:16.929+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:16.947 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:16.947 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:16.955 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:17.030 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:17.029+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:17.064 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.468392209999998, 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:17.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.069 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.460060538, 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:17.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.456719683999999, 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.074 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:17.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:17.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.429264855, 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:17.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.395299538000003, 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:17.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.314331952000003, 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:17.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:17.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.314310114000001, 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:17.092 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:17.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:17.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.423964814000001, 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:17.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.422421949, 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.422400792000001, 2024-09-18T01:08:17.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:17.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.422381978000001, 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.111 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:17.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:17.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.968875182000001, 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:08:17.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:08:17.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:08:17.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.121 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.968863802000001, 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:17.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:17.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.968849882000001, 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:17.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:17.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.492315297000001, 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.134 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:08:17.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:08:17.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.492303818, 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.141 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:17.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:17.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:17.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.337111959000001, 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:17.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:17.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.400105960000001, 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:17.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:17.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.3361554470000003, 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:17.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:17.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.1984118449999999, 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:17.167 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:17.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:17.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:17.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:17.173 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:17.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:17.175 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:17.174+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:17.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:17.276 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:17.275+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:17.312 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:17.320 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:17.320 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:17.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:17.330 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:17.330 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:17.345 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:17.345 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:17.360 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:17.377 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:17.377 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:17.377 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:17.376+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:17.393 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:17.393 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:17.409 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:17.410 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:17.426 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:17.426 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:17.473 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:17.473 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:17.477 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:17.476+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:17.482 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:17.579 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:17.578+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:17.626 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:17.626 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:17.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:17.680 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:17.679+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:17.742 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:17.742 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:17.744 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:08:17.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:17.794 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:17.793+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:17.881 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:17.881+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:17.896 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:17.896 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:17.905 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:17.983 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:17.982+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:18.042 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.042 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:18.097 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:18.096+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:18.161 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.161 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.166 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:18.207 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:18.206+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:18.286 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:18.285+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:18.309 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.309 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.318 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:18.402 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:18.401+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:18.463 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.463 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:18.514 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:18.513+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:18.580 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.580 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.589 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:18.589 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:18.587+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:18.690 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:18.689+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:18.691 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.691 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:18.746 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:08:18.790 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:18.790+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:18.813 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:18.813 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:18.823 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:18.891 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:18.890+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:18.892 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-18T01:08:18.993 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:18.992+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:19.008 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:19.008 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:19.017 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:19.102 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:19.101+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:19.117 INFO:teuthology.orchestra.run.smithi120.stdout:1170 read 0xcabf7d4 thru 0xcac9c54 (0xa481 bytes) 2024-09-18T01:08:19.118 INFO:teuthology.orchestra.run.smithi120.stdout:1171 write 0x4e93ee7 thru 0x4e9e2e6 (0xa400 bytes) 2024-09-18T01:08:19.137 INFO:teuthology.orchestra.run.smithi120.stdout:1173 writesame 0xd4c981e thru 0xd4d3430 data_size 0x3d(0x9c13 bytes) 2024-09-18T01:08:19.137 INFO:teuthology.orchestra.run.smithi120.stdout:1174 read 0xad7988 thru 0xade8db (0x6f54 bytes) 2024-09-18T01:08:19.137 INFO:teuthology.orchestra.run.smithi120.stdout:1175 write 0x7536cc5 thru 0x7538cb1 (0x1fed bytes) 2024-09-18T01:08:19.156 INFO:teuthology.orchestra.run.smithi120.stdout:1176 trunc from 0xd4d3431 to 0x3cd9ba 2024-09-18T01:08:19.171 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:19.171 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:19.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:19.195 INFO:teuthology.orchestra.run.smithi120.stdout:1178 punch from 0x2efb45 to 0x2fa0b4, (0xa56f bytes) 2024-09-18T01:08:19.195 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:19.193+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:19.216 INFO:teuthology.orchestra.run.smithi120.stdout:1180 writesame 0xcc02521 thru 0xcc0400a data_size 0x35(0x1aea bytes) 2024-09-18T01:08:19.216 INFO:teuthology.orchestra.run.smithi120.stdout:1182 read 0xab626f8 thru 0xab67187 (0x4a90 bytes) 2024-09-18T01:08:19.216 INFO:teuthology.orchestra.run.smithi120.stdout:1184 punch from 0x5592ce8 to 0x559bce9, (0x9001 bytes) 2024-09-18T01:08:19.218 INFO:teuthology.orchestra.run.smithi120.stdout:1187 trunc from 0xcc0400b to 0xa3552a0 2024-09-18T01:08:19.224 INFO:teuthology.orchestra.run.smithi120.stdout:1188 read 0x8912364 thru 0x89222a0 (0xff3d bytes) 2024-09-18T01:08:19.234 INFO:teuthology.orchestra.run.smithi120.stdout:1189 writesame 0xc222296 thru 0xc228101 data_size 0x2(0x5e6c bytes) 2024-09-18T01:08:19.242 INFO:teuthology.orchestra.run.smithi120.stdout:1191 writesame 0xd14b7ce thru 0xd150c4b data_size 0x46(0x547e bytes) 2024-09-18T01:08:19.242 INFO:teuthology.orchestra.run.smithi120.stdout:1193 punch from 0x6e61dbc to 0x6e66d13, (0x4f57 bytes) 2024-09-18T01:08:19.242 INFO:teuthology.orchestra.run.smithi120.stdout:1194 punch from 0x93bc25 to 0x943189, (0x7564 bytes) 2024-09-18T01:08:19.242 INFO:teuthology.orchestra.run.smithi120.stdout:1195 read 0x3574ee7 thru 0x357b27c (0x6396 bytes) 2024-09-18T01:08:19.257 INFO:teuthology.orchestra.run.smithi120.stdout:1196 trunc from 0xd150c4c to 0x169752 2024-09-18T01:08:19.296 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:19.295+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:19.296 INFO:teuthology.orchestra.run.smithi120.stdout:1197 writesame 0xc020a5d thru 0xc02d64c data_size 0x68(0xcbf0 bytes) 2024-09-18T01:08:19.296 INFO:teuthology.orchestra.run.smithi120.stdout:1202 read 0x60e9216 thru 0x60f3564 (0xa34f bytes) 2024-09-18T01:08:19.301 INFO:teuthology.orchestra.run.smithi120.stdout:1203 trunc from 0xc02d64d to 0x792a0ac 2024-09-18T01:08:19.313 INFO:teuthology.orchestra.run.smithi120.stdout:1204 trunc from 0x792a0ac to 0xc4d3f11 2024-09-18T01:08:19.315 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:19.315 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:19.318 INFO:teuthology.orchestra.run.smithi120.stdout:1206 writesame 0x487bf1b thru 0x4881eee data_size 0x2(0x5fd4 bytes) 2024-09-18T01:08:19.325 INFO:teuthology.orchestra.run.smithi120.stdout:1207 punch from 0xa848e3a to 0xa850713, (0x78d9 bytes) 2024-09-18T01:08:19.326 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:19.326 INFO:teuthology.orchestra.run.smithi120.stdout:1209 write 0x358db49 thru 0x3590d14 (0x31cc bytes) 2024-09-18T01:08:19.329 INFO:teuthology.orchestra.run.smithi120.stdout:1210 write 0xb55b55c thru 0xb5686aa (0xd14f bytes) 2024-09-18T01:08:19.332 INFO:teuthology.orchestra.run.smithi120.stdout:1211 read 0x33caec1 thru 0x33d27e2 (0x7922 bytes) 2024-09-18T01:08:19.332 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.332 INFO:teuthology.orchestra.run.smithi120.stdout:1214 write 0xb006c34 thru 0xb00d232 (0x65ff bytes) 2024-09-18T01:08:19.336 INFO:teuthology.orchestra.run.smithi120.stdout:1215 read 0x8818691 thru 0x88208bb (0x822b bytes) 2024-09-18T01:08:19.336 INFO:teuthology.orchestra.run.smithi120.stdout:1216 read 0x24dade5 thru 0x24e45b3 (0x97cf bytes) 2024-09-18T01:08:19.336 INFO:teuthology.orchestra.run.smithi120.stdout:1219 read 0xc1e4dc2 thru 0xc1e4f1e (0x15d bytes) 2024-09-18T01:08:19.336 INFO:teuthology.orchestra.run.smithi120.stdout:1220 write 0x73facaf thru 0x740799f (0xccf1 bytes) 2024-09-18T01:08:19.341 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.341 INFO:teuthology.orchestra.run.smithi120.stdout:1223 read 0x573a97b thru 0x5743467 (0x8aed bytes) 2024-09-18T01:08:19.341 INFO:teuthology.orchestra.run.smithi120.stdout:1224 write 0x7a2dc20 thru 0x7a2dc39 (0x1a bytes) 2024-09-18T01:08:19.344 INFO:teuthology.orchestra.run.smithi120.stdout:1225 trunc from 0xc4d3f11 to 0xb415ea9 2024-09-18T01:08:19.351 INFO:teuthology.orchestra.run.smithi120.stdout:1227 writesame 0x24f4644 thru 0x24f801a data_size 0x11(0x39d7 bytes) 2024-09-18T01:08:19.355 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.355 INFO:teuthology.orchestra.run.smithi120.stdout:1230 write 0x59f5f23 thru 0x5a03a15 (0xdaf3 bytes) 2024-09-18T01:08:19.368 INFO:teuthology.orchestra.run.smithi120.stdout:1231 trunc from 0xb415ea9 to 0x793ec87 2024-09-18T01:08:19.386 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.386 INFO:teuthology.orchestra.run.smithi120.stdout:1234 punch from 0x52e01b0 to 0x52e3208, (0x3058 bytes) 2024-09-18T01:08:19.386 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.398 INFO:teuthology.orchestra.run.smithi120.stdout:1236 write 0x9fc46be thru 0x9fcf3ed (0xad30 bytes) 2024-09-18T01:08:19.399 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:19.398+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:19.400 INFO:teuthology.orchestra.run.smithi120.stdout:1238 punch from 0x22a0c72 to 0x22a370a, (0x2a98 bytes) 2024-09-18T01:08:19.407 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:19.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:08:19.411 INFO:teuthology.orchestra.run.smithi120.stdout:1241 write 0xc5af5c0 thru 0xc5bb91e (0xc35f bytes) 2024-09-18T01:08:19.417 INFO:teuthology.orchestra.run.smithi120.stdout:1242 read 0x23460af thru 0x234a898 (0x47ea bytes) 2024-09-18T01:08:19.417 INFO:teuthology.orchestra.run.smithi120.stdout:1244 punch from 0x373350 to 0x37e312, (0xafc2 bytes) 2024-09-18T01:08:19.417 INFO:teuthology.orchestra.run.smithi120.stdout:1245 writesame 0x3b6469c thru 0x3b654b2 data_size 0x1(0xe17 bytes) 2024-09-18T01:08:19.432 INFO:teuthology.orchestra.run.smithi120.stdout:1246 trunc from 0xc5bb91f to 0x55e5ed4 2024-09-18T01:08:19.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:19.450 INFO:teuthology.orchestra.run.smithi120.stdout:1247 read 0xde6e22 thru 0xded7fa (0x69d9 bytes) 2024-09-18T01:08:19.450 INFO:teuthology.orchestra.run.smithi120.stdout:1248 read 0x41af450 thru 0x41b65af (0x7160 bytes) 2024-09-18T01:08:19.453 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:19.453 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:19.458 INFO:teuthology.orchestra.run.smithi120.stdout:1250 write 0x6f3a110 thru 0x6f41cac (0x7b9d bytes) 2024-09-18T01:08:19.464 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1251 punch from 0x5bbb595 to 0x5bc2bf6, (0x7661 bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1253 read 0x5fe7a33 thru 0x5fe8b84 (0x1152 bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1255 read 0x24b5d50 thru 0x24bf8ab (0x9b5c bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1257 punch from 0x44b3498 to 0x44b7477, (0x3fdf bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1259 read 0x6629c7d thru 0x66312f3 (0x7677 bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1260 punch from 0xe02f67 to 0xe0f164, (0xc1fd bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:1261 read 0x6995a4a thru 0x6997466 (0x1a1d bytes) 2024-09-18T01:08:19.465 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:19.472 INFO:teuthology.orchestra.run.smithi120.stdout:1263 writesame 0x7cfaaa3 thru 0x7d0538f data_size 0x1f(0xa8ed bytes) 2024-09-18T01:08:19.473 INFO:teuthology.orchestra.run.smithi120.stdout:1265 clone 15 order 19 su 262144 sc 2 2024-09-18T01:08:19.499 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:19.498+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:19.576 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:19.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:19.597 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:19.599 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:19.598+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:19.616 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:19.633 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:19.650 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:19.672 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:19.689 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:19.700 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:19.699+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.763023212999997, 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.635095173000003, 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.584721113000001, 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.494827897, 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.493232149000001, 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.492421530000001, 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2813 3.4 e28/23 3:248773b5:::rbd_data.11769efc726f.00000000000000ad:head v 28'13, mlcod=28'11)", 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.441874036999998, 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033191399999999999, 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260874+0000", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260875+0000", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260907+0000", 2024-09-18T01:08:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2240999999999998e-05 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260922+0000", 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4453999999999999e-05 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260930+0000", 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.260991+0000", 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1734000000000002e-05 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.261012+0000", 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1060000000000002e-05 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264143+0000", 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00313102 2024-09-18T01:08:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.264193+0000", 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9873999999999997e-05 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.109048719, 2024-09-18T01:08:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.545448661999998, 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.527687575999998, 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.525030872999999, 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 25.942383329999998, 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 25.528069871, 2024-09-18T01:08:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.0263708489999992, 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.0247939620000004, 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9950463720000009, 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9494032109999999, 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39974 3.4 e63/23 3:232e8047:::rbd_data.11762bc2f01c.000000000000003d:head v 63'122, mlcod=61'120)", 2024-09-18T01:08:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.9319129739999994, 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034513289999999999, 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770835+0000", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770836+0000", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3540000000000001e-06 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770967+0000", 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130853 2024-09-18T01:08:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770988+0000", 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0514e-05 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.770998+0000", 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771055+0000", 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7376000000000001e-05 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.771073+0000", 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8406999999999998e-05 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774238+0000", 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003164287 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.774286+0000", 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8838000000000002e-05 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.8396668859999998, 2024-09-18T01:08:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.8360134559999999, 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:19.798 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:19.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:19.804 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:19.800+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:19.902 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:19.900+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:19.943 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:19.943 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:19.952 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:20.002 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:20.001+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:20.055 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:20.056 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:20.056 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:20.056 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:20.056 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:20.064 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:20.108 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:20.107+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:20.169 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38805 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2424832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5988856330000001, 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244350999999999, 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38805 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569048+0000", 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569057+0000", 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4530000000000008e-06 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569060+0000", 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0170000000000001e-06 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611025+0000", 2024-09-18T01:08:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041964317000000001 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611073+0000", 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8037000000000001e-05 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611288+0000", 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215596 2024-09-18T01:08:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38806 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5988658079999993, 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042508381999999997, 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38806 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4400000000000001e-07 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569065+0000", 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.206e-06 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569068+0000", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2890000000000002e-06 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569071+0000", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611316+0000", 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244918999999999 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611360+0000", 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4239000000000001e-05 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611572+0000", 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212186 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38807 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5988560550000006, 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042802420000000001, 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38807 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569074+0000", 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569075+0000", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569078+0000", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9749999999999999e-06 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569080+0000", 2024-09-18T01:08:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611602+0000", 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042521681999999998 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611650+0000", 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8179000000000001e-05 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611876+0000", 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022609299999999999 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38808 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5988462059999993, 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043110682999999997, 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38808 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569084+0000", 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569085+0000", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569088+0000", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569090+0000", 2024-09-18T01:08:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3120000000000001e-06 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611901+0000", 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042810991999999999 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611955+0000", 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3983000000000001e-05 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612194+0000", 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239182 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38809 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 475136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5988278810000001, 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043389637000000002, 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38809 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569122+0000", 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0254999999999998e-05 2024-09-18T01:08:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569127+0000", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569130+0000", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.016e-06 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612224+0000", 2024-09-18T01:08:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043094406000000002 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612275+0000", 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1054000000000003e-05 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612491+0000", 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215987 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38810 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987950350000002, 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043652712000000003, 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38810 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569135+0000", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569136+0000", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.083e-06 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569141+0000", 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-18T01:08:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569143+0000", 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3369999999999998e-06 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612521+0000", 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043377538 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612566+0000", 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5485000000000002e-05 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612787+0000", 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221158 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38811 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987837539999994, 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043930856999999997, 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38811 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569147+0000", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0449999999999999e-06 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569150+0000", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569152+0000", 2024-09-18T01:08:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612830+0000", 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043677597999999998 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612880+0000", 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0219999999999997e-05 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613077+0000", 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019646600000000001 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38812 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987745449999995, 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044183260000000002, 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38812 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569156+0000", 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999991e-07 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569160+0000", 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3759999999999999e-06 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569162+0000", 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613107+0000", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043945166000000001 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613153+0000", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613338+0000", 2024-09-18T01:08:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018564800000000001 2024-09-18T01:08:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38813 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1130496~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987650769999995, 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044519186000000002, 2024-09-18T01:08:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38813 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569165+0000", 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569166+0000", 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2399999999999996e-07 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569168+0000", 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9229999999999998e-06 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569178+0000", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6479999999999998e-06 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613370+0000", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044191800000000003 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613416+0000", 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6431999999999999e-05 2024-09-18T01:08:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613684+0000", 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026725300000000001 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38814 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987482459999995, 2024-09-18T01:08:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044848796000000003, 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38814 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569183+0000", 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.096e-06 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569187+0000", 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569191+0000", 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1930000000000002e-06 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613735+0000", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044543886999999997 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613784+0000", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9858999999999998e-05 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614030+0000", 2024-09-18T01:08:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024569700000000002 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38815 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1458176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987361349999993, 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045101439, 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38815 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569194+0000", 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569195+0000", 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0389999999999999e-06 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569198+0000", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9210000000000001e-06 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569200+0000", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.367e-06 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614061+0000", 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044861375000000002 2024-09-18T01:08:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614104+0000", 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3142000000000001e-05 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614295+0000", 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190383 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38816 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1622016~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987270250000005, 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045464629999999999, 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38816 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569203+0000", 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569206+0000", 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.754e-06 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569211+0000", 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.109e-06 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569213+0000", 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.379e-06 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614331+0000", 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045118395999999998 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614380+0000", 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614667+0000", 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028724000000000002 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38817 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1785856~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987136320000008, 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045799302, 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38817 2024-09-18T01:08:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569217+0000", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.035e-06 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569221+0000", 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.839e-06 2024-09-18T01:08:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569223+0000", 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3300000000000001e-06 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614711+0000", 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045487811000000003 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614772+0000", 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0729000000000001e-05 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615015+0000", 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024334399999999999 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38818 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5987035820000006, 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046120525000000002, 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38818 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569230+0000", 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0260000000000001e-06 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569234+0000", 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569237+0000", 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3750000000000001e-06 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615046+0000", 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045809435000000003 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615096+0000", 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9654999999999998e-05 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615346+0000", 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025073700000000001 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38819 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2113536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.598687795, 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046434472999999997, 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38819 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569243+0000", 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.291e-06 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569247+0000", 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.031e-06 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569250+0000", 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615395+0000", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046145046000000002 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615437+0000", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2407999999999998e-05 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615676+0000", 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239137 2024-09-18T01:08:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38820 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2277376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5986768970000007, 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046745223000000002, 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38820 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569254+0000", 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-07 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569258+0000", 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6489999999999999e-06 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569261+0000", 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.322e-06 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615704+0000", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046442986999999998 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615764+0000", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9871000000000003e-05 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615998+0000", 2024-09-18T01:08:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023420099999999999 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38821 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2441216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5986660449999999, 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047015754999999999, 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38821 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569264+0000", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569265+0000", 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8599999999999996e-07 2024-09-18T01:08:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569269+0000", 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8800000000000001e-06 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569271+0000", 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616017+0000", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046746593000000003 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616061+0000", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3894999999999999e-05 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616279+0000", 2024-09-18T01:08:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000217875 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38822 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5986559729999996, 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047344570000000002, 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38822 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569275+0000", 2024-09-18T01:08:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.026e-06 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569278+0000", 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999999e-06 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569284+0000", 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.071e-06 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616318+0000", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047033859999999997 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616374+0000", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5636000000000001e-05 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616618+0000", 2024-09-18T01:08:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024412699999999999 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38823 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5946794000000004, 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045935959999999998, 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38823 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573251+0000", 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573255+0000", 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2520000000000001e-06 2024-09-18T01:08:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573266+0000", 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0958000000000001e-05 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573271+0000", 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1229999999999999e-06 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616299+0000", 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043027351999999998 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616351+0000", 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2284000000000003e-05 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616681+0000", 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033019400000000002 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619078+0000", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023970749999999998 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619093+0000", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4338999999999999e-05 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619143+0000", 2024-09-18T01:08:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0346000000000002e-05 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619186+0000", 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3226999999999998e-05 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38824 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5893400510000006, 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042771339999999998, 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38824 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578590+0000", 2024-09-18T01:08:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578594+0000", 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0609999999999997e-06 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578605+0000", 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0477e-05 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578611+0000", 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9540000000000001e-06 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617084+0000", 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.038473680000000003 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617135+0000", 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0139000000000002e-05 2024-09-18T01:08:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617316+0000", 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181082 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621302+0000", 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039860210000000002 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621309+0000", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6490000000000008e-06 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621336+0000", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6828000000000001e-05 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621361+0000", 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4621999999999999e-05 2024-09-18T01:08:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:20.275 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:20.276 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:20.277 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:20.275+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:20.377 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:20.377+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:20.383 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:20.383 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:20.383 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:20.383 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:20.392 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:20.478 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:20.477+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:20.494 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:20.495 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:20.495 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:20.495 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:20.495 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:20.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:20.551 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:08:20.570 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:08:20.611 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:20.610+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.061587873000001, 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.053256201000004, 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.049915347000002, 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.022460518000003, 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.988495200999999, 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.907527614999999, 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.907505776999997, 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.017160477000001, 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.015617612, 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.015596455000001, 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.015577641, 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.562070845000001, 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:08:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:08:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.562059465000001, 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.562045545, 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.085510960000001, 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:08:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:08:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.085499480999999, 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.930307622000001, 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.993301623000001, 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.9293511100000007, 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7916075080000002, 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:20.773 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:20.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:08:20.775 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:20.773+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:20.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:20.876 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:20.875+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:20.880 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:20.880 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:20.889 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:20.977 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:20.976+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:20.991 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:20.991 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:20.991 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:20.991 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:20.991 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:20.999 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:21.078 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:21.077+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:21.104 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:21.104 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:21.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079585729000001, 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.105 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:21.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:21.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079543577999999, 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.110 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:21.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079510149999997, 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.114 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:21.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.07947265, 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:21.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079451073000001, 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:21.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:21.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079431933000002, 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:21.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:21.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079411542000003, 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:21.133 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:21.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:21.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079382142, 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:21.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.142 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079353351999998, 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:21.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079332657000002, 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079313173000003, 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:21.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079279262, 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:21.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.079243263000002, 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:21.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:21.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.994795291000003, 2024-09-18T01:08:21.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:21.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:21.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.994785082, 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:21.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:21.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.494383855999999, 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:21.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:21.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.478897687, 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.471006967000001, 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.444937963999998, 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.439769171999998, 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:21.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.201 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:21.202 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:21.202 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:21.201+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:21.303 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:21.302+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:21.346 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:21.346 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:21.346 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:21.346 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:21.355 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:21.442 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:21.441+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:21.501 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:21.501 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:21.501 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:21.501 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:21.501 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:21.505 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:21.504+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:21.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:21.552 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:08:21.552 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:08:21.569 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:08:21.605 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:21.605+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:21.657 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:21.657 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:21.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:21.657 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:21.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.573252951000001, 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:21.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:21.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.469938655, 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:21.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.993262102000003, 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:21.670 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:21.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:21.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.676 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.909329864, 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:21.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:21.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.907320730999999, 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13185 3.a 3:51a12e8c:::rbd_data.117611b60058.00000000000000ee:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.506617181000003, 2024-09-18T01:08:21.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00605274, 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13185 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:21.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148174+0000", 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148179+0000", 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148182+0000", 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2540000000000001e-06 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148734+0000", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055139499999999999 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148870+0000", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013613699999999999 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.154225+0000", 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053546829999999998 2024-09-18T01:08:21.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13241 3.10 3:090d2a4d:::rbd_data.117611b60058.0000000000000149:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.482894510000001, 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005514878, 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13241 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:21.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171896+0000", 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171908+0000", 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2401e-05 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171918+0000", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.042e-05 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171925+0000", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171967+0000", 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1257000000000002e-05 2024-09-18T01:08:21.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.172085+0000", 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011883999999999999 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.177409+0000", 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053241210000000002 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:15123 3.1e e43/23 3:7a2bedf1:::rbd_data.1176492bc9ca.0000000000000000:head v 43'60, mlcod=42'58)", 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.089910670999998, 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061319520000000004, 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:21.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564879+0000", 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565612+0000", 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073299200000000002 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565627+0000", 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5634e-05 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565638+0000", 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0462000000000001e-05 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565707+0000", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9402999999999999e-05 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565730+0000", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3108999999999998e-05 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570965+0000", 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052347380000000001 2024-09-18T01:08:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571010+0000", 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5187e-05 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.676097005999999, 2024-09-18T01:08:21.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:21.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:21.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18543 3.a 3:5239904c:::rbd_data.11762d95e74a.0000000000000038:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.615267865, 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059139489999999999, 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18543 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039523+0000", 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.325e-06 2024-09-18T01:08:21.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039529+0000", 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039541+0000", 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2342e-05 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039548+0000", 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.99e-06 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039592+0000", 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3915000000000002e-05 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039646+0000", 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3668000000000002e-05 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.040432+0000", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000786288 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.044920+0000", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004487624 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045325+0000", 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000405308 2024-09-18T01:08:21.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045381+0000", 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5550000000000002e-05 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045435+0000", 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4657000000000002e-05 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27452 3.a 3:52657a03:::rbd_data.1176492bc9ca.0000000000000045:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.756111267000001, 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057803079999999996, 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.714 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27452 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898679+0000", 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898682+0000", 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8060000000000001e-06 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898694+0000", 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1652e-05 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898700+0000", 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4969999999999998e-06 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898807+0000", 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010601 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898813+0000", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904378+0000", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055656710000000003 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904458+0000", 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9935e-05 2024-09-18T01:08:21.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27469 3.a 3:52bb01ff:::rbd_data.1176492bc9ca.0000000000000056:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.756069979999999, 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058442279999999999, 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27469 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898720+0000", 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.37e-07 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898723+0000", 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898730+0000", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898735+0000", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898843+0000", 2024-09-18T01:08:21.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010823900000000001 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898848+0000", 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8230000000000002e-06 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904484+0000", 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056357589999999997 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904563+0000", 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9746000000000005e-05 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27546 3.a 3:5585dcf7:::rbd_data.1176492bc9ca.00000000000000a3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.755565597, 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055778030000000001, 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27546 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899226+0000", 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5000000000000002e-06 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899234+0000", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1679999999999998e-06 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899239+0000", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999998e-06 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899338+0000", 2024-09-18T01:08:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.912e-05 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899344+0000", 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904713+0000", 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053690400000000003 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904801+0000", 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7967999999999995e-05 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27837 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.505353241999998, 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056104429999999997, 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27837 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149437+0000", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999997e-07 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149449+0000", 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2368000000000001e-05 2024-09-18T01:08:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149462+0000", 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3277e-05 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149471+0000", 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3799999999999994e-06 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149517+0000", 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5838999999999999e-05 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149607+0000", 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0650999999999997e-05 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149715+0000", 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107345 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154757+0000", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005042571 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154948+0000", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019090200000000001 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155005+0000", 2024-09-18T01:08:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6504e-05 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155046+0000", 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1696000000000002e-05 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28147 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.257813809000002, 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006127146, 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28147 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396977+0000", 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4050000000000001e-06 2024-09-18T01:08:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396988+0000", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1046000000000001e-05 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397002+0000", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3895e-05 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397008+0000", 2024-09-18T01:08:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397064+0000", 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5961999999999997e-05 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397147+0000", 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3233999999999996e-05 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397295+0000", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014745700000000001 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.400032+0000", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027374460000000001 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403019+0000", 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029870600000000001 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403061+0000", 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1999000000000002e-05 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403102+0000", 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1072999999999998e-05 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.149026079999999, 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.148998298999999, 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.147579206, 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.019517015000002, 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:40414 3.a e63/51 3:55d9e024:::rbd_data.11762bc2f01c.000000000000004f:head v 63'84, mlcod=63'82)", 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.827547457, 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060175339999999997, 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827243+0000", 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.372e-06 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827327+0000", 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3904000000000002e-05 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827346+0000", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9048000000000001e-05 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827355+0000", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9129999999999992e-06 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827438+0000", 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2930999999999996e-05 2024-09-18T01:08:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827465+0000", 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7376999999999999e-05 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.833209+0000", 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057439659999999997 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.833259+0000", 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0022999999999999e-05 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.770 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-18T01:08:21.771 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:21.772 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:21.770+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:21.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:21.872 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:21.871+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:21.915 INFO:teuthology.orchestra.run.smithi120.stdout:1266 read 0x271ef07 thru 0x2721f13 (0x300d bytes) 2024-09-18T01:08:21.916 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:21.916 INFO:teuthology.orchestra.run.smithi120.stdout:1269 write 0x45c4ddb thru 0x45cef30 (0xa156 bytes) 2024-09-18T01:08:21.917 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:49357 3.12 e74/23 3:4ff79aae:::rbd_header.117667234fca:head v 74'186, mlcod=74'184)", 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:21.917041+0000", 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00023604299999999999, 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254913, 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917041+0000", 2024-09-18T01:08:21.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917041+0000", 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917043+0000", 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0130000000000001e-06 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917050+0000", 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8800000000000008e-06 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917063+0000", 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2678e-05 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917069+0000", 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1090000000000003e-06 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917119+0000", 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0328999999999999e-05 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:21.917136+0000", 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.679e-05 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-18T01:08:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-18T01:08:21.922 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:21.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:21.927 INFO:teuthology.orchestra.run.smithi120.stdout:1271 punch from 0x70f433 to 0x71a7b9, (0xb386 bytes) 2024-09-18T01:08:21.928 INFO:teuthology.orchestra.run.smithi120.stdout:1272 read 0x4b9cbc9 thru 0x4b9dad9 (0xf11 bytes) 2024-09-18T01:08:21.928 INFO:teuthology.orchestra.run.smithi120.stdout:1273 read 0x5f86cea thru 0x5f958a5 (0xebbc bytes) 2024-09-18T01:08:21.928 INFO:teuthology.orchestra.run.smithi120.stdout:1275 read 0x20cf8a5 thru 0x20d0b4f (0x12ab bytes) 2024-09-18T01:08:21.928 INFO:teuthology.orchestra.run.smithi120.stdout:1276 read 0x4dd2490 thru 0x4dd62c8 (0x3e39 bytes) 2024-09-18T01:08:21.929 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:21.929 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:21.932 INFO:teuthology.orchestra.run.smithi120.stdout:1277 write 0x8445a1a thru 0x8447f50 (0x2537 bytes) 2024-09-18T01:08:21.935 INFO:teuthology.orchestra.run.smithi120.stdout:1278 read 0x3d35f1 thru 0x3d774d (0x415d bytes) 2024-09-18T01:08:21.935 INFO:teuthology.orchestra.run.smithi120.stdout:1279 write 0x63286d1 thru 0x633668e (0xdfbe bytes) 2024-09-18T01:08:21.941 INFO:teuthology.orchestra.run.smithi120.stdout:1281 read 0x251eb45 thru 0x252271a (0x3bd6 bytes) 2024-09-18T01:08:21.942 INFO:teuthology.orchestra.run.smithi120.stdout:1282 punch from 0x39fc21c to 0x3a0b694, (0xf478 bytes) 2024-09-18T01:08:21.942 INFO:teuthology.orchestra.run.smithi120.stdout:1283 write 0x9b0561 thru 0x9bbe78 (0xb918 bytes) 2024-09-18T01:08:21.944 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:21.946 INFO:teuthology.orchestra.run.smithi120.stdout:1285 trunc from 0x8447f51 to 0x6e65f40 2024-09-18T01:08:21.946 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:21.947 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:21.965 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:21.965 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:21.971 INFO:teuthology.orchestra.run.smithi120.stdout:1287 writesame 0xe17e340 thru 0xe17f89b data_size 0x2(0x155c bytes) 2024-09-18T01:08:21.971 INFO:teuthology.orchestra.run.smithi120.stdout:1289 writesame 0x1dbb4a3 thru 0x1dc5f77 data_size 0x1(0xaad5 bytes) 2024-09-18T01:08:21.973 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:21.972+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:21.982 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:21.982 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:21.985 INFO:teuthology.orchestra.run.smithi120.stdout:1290 write 0xad30433 thru 0xad362e8 (0x5eb6 bytes) 2024-09-18T01:08:21.989 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:21.989 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:21.989 INFO:teuthology.orchestra.run.smithi120.stdout:1293 read 0x652ded3 thru 0x652e9cf (0xafd bytes) 2024-09-18T01:08:22.000 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:22.001 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:22.005 INFO:teuthology.orchestra.run.smithi120.stdout:1294 trunc from 0xe17f89c to 0x6c89f6 2024-09-18T01:08:22.018 INFO:teuthology.orchestra.run.smithi120.stdout:1295 read 0x5bdb4a thru 0x5cd6fe (0xfbb5 bytes) 2024-09-18T01:08:22.021 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:22.021 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:22.034 INFO:teuthology.orchestra.run.smithi120.stdout:1297 trunc from 0x6c89f6 to 0xde9665c 2024-09-18T01:08:22.038 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:22.038 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:22.039 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:22.039 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:22.039 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:22.040 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:22.040 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:22.041 INFO:teuthology.orchestra.run.smithi120.stdout:1298 punch from 0x1bb2693 to 0x1bb8a15, (0x6382 bytes) 2024-09-18T01:08:22.041 INFO:teuthology.orchestra.run.smithi120.stdout:1299 punch from 0x9aab1f0 to 0x9ab59e0, (0xa7f0 bytes) 2024-09-18T01:08:22.041 INFO:teuthology.orchestra.run.smithi120.stdout:1300 trunc from 0xde9665c to 0xdf9b3e6 2024-09-18T01:08:22.045 INFO:teuthology.orchestra.run.smithi120.stdout:1302 read 0x3a6e9d4 thru 0x3a7e0ca (0xf6f7 bytes) 2024-09-18T01:08:22.045 INFO:teuthology.orchestra.run.smithi120.stdout:1303 read 0x657b375 thru 0x657cf05 (0x1b91 bytes) 2024-09-18T01:08:22.045 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:22.045 INFO:teuthology.orchestra.run.smithi120.stdout:1306 read 0x1cbe5de thru 0x1cc996d (0xb390 bytes) 2024-09-18T01:08:22.045 INFO:teuthology.orchestra.run.smithi120.stdout:1307 clone 16 order 24 su 4194304 sc 10 2024-09-18T01:08:22.048 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:22.058 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:22.058 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:22.110 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:22.109+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:22.152 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.152 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:22.206 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:22.205+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:22.264 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.264 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.273 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:22.314 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:22.313+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:22.377 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:22.376+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:22.417 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.417 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:22.511 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:22.510+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:22.569 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.569 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:22.618 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:22.617+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:22.679 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:22.678+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:22.720 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.720 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.729 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:22.754 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:08:22.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:08:22.811 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:22.810+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:22.839 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.840 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:22.849 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:22.891 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:22.890+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:22.982 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:22.982+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:22.996 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:22.996 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:23.083 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:23.082+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:23.149 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.149 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.157 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:23.185 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:23.184+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:23.260 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.283 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.284 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:23.285 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:23.284+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:23.386 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:23.385+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:23.387 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.387 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.395 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:23.487 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:23.486+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.507 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:23.569 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:08:23.588 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:23.587+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:23.612 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.612 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.621 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:23.705 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:23.704+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:23.755 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:08:23.777 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:08:23.777 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.778 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.790 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:23.832 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:23.831+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:23.891 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:23.890+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:23.901 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:23.902 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:23.910 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:23.995 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:23.994+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:24.054 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:24.054 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:24.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:24.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:24.113 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:24.111+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:24.188 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:24.197 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:24.196+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:24.205 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:24.207 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:24.207 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:24.216 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:24.220 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:24.221 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:24.236 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:24.236 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:24.252 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:24.253 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:24.269 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:24.285 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:24.286 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:24.301 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:24.299+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:24.302 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:24.302 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:24.320 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:24.320 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:24.321 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:24.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:24.402 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:24.401+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:24.433 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:24.433 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:24.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:24.506 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:24.505+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:24.542 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:08:24.547 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:24.547 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:24.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:24.547 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.522103573000003, 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:24.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:24.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.522061422, 2024-09-18T01:08:24.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:24.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:24.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.522027993999998, 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:24.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521990494000001, 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:24.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:24.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521968917000002, 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:24.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521949777000003, 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521929385999996, 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:24.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:24.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521899986000001, 2024-09-18T01:08:24.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:24.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521871195999999, 2024-09-18T01:08:24.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:24.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521850501000003, 2024-09-18T01:08:24.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521831016999997, 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.597 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:24.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.601 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521797106000001, 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.602 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:24.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:24.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.521761107000003, 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.607 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:24.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:24.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.437313134999997, 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:24.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.437302926000001, 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.9369017, 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:24.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.921415531000001, 2024-09-18T01:08:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.913524811000002, 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.887455807999999, 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.882287015999999, 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:24.650 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:08:24.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:24.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:08:24.653 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:24.649+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:24.701 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-18T01:08:24.752 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:24.751+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:24.804 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:24.804 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:24.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:24.844 INFO:teuthology.orchestra.run.smithi120.stdout:1308 punch from 0x30ba23 to 0x316a9a, (0xb077 bytes) 2024-09-18T01:08:24.853 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:24.851+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:24.855 INFO:teuthology.orchestra.run.smithi120.stdout:1310 trunc from 0xdf9b3e6 to 0x84d971c 2024-09-18T01:08:24.861 INFO:teuthology.orchestra.run.smithi120.stdout:1312 write 0x3ab6bfb thru 0x3ac6217 (0xf61d bytes) 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:1317 punch from 0x471b9b5 to 0x4724c1a, (0x9265 bytes) 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:1318 read 0x5bc475d thru 0x5bca4cc (0x5d70 bytes) 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:1319 read 0x38c8d3b thru 0x38ca518 (0x17de bytes) 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:1320 punch from 0x2fb2d20 to 0x2fbfb8e, (0xce6e bytes) 2024-09-18T01:08:24.865 INFO:teuthology.orchestra.run.smithi120.stdout:1321 punch from 0x732b6b5 to 0x732c5fe, (0xf49 bytes) 2024-09-18T01:08:24.873 INFO:teuthology.orchestra.run.smithi120.stdout:1322 write 0xaa1d095 thru 0xaa1fc81 (0x2bed bytes) 2024-09-18T01:08:24.886 INFO:teuthology.orchestra.run.smithi120.stdout:1323 write 0xcadca22 thru 0xcae7bd9 (0xb1b8 bytes) 2024-09-18T01:08:24.890 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.890 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.890 INFO:teuthology.orchestra.run.smithi120.stdout:1326 write 0xa2b5194 thru 0xa2b6c04 (0x1a71 bytes) 2024-09-18T01:08:24.892 INFO:teuthology.orchestra.run.smithi120.stdout:1327 read 0x474179d thru 0x474264d (0xeb1 bytes) 2024-09-18T01:08:24.892 INFO:teuthology.orchestra.run.smithi120.stdout:1328 punch from 0x15ef182 to 0x15f7e70, (0x8cee bytes) 2024-09-18T01:08:24.892 INFO:teuthology.orchestra.run.smithi120.stdout:1330 punch from 0x70245d0 to 0x7029ca9, (0x56d9 bytes) 2024-09-18T01:08:24.892 INFO:teuthology.orchestra.run.smithi120.stdout:1331 write 0x1a06da5 thru 0x1a13118 (0xc374 bytes) 2024-09-18T01:08:24.900 INFO:teuthology.orchestra.run.smithi120.stdout:1332 writesame 0x8a0545c thru 0x8a0a74d data_size 0x3(0x52f2 bytes) 2024-09-18T01:08:24.906 INFO:teuthology.orchestra.run.smithi120.stdout:1334 read 0x4df1b69 thru 0x4df2084 (0x51c bytes) 2024-09-18T01:08:24.906 INFO:teuthology.orchestra.run.smithi120.stdout:1335 write 0x829edcf thru 0x82ad549 (0xe77b bytes) 2024-09-18T01:08:24.920 INFO:teuthology.orchestra.run.smithi120.stdout:1338 write 0xde9f732 thru 0xdead865 (0xe134 bytes) 2024-09-18T01:08:24.924 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.924 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.924 INFO:teuthology.orchestra.run.smithi120.stdout:1342 write 0xd8c6019 thru 0xd8cc9d0 (0x69b8 bytes) 2024-09-18T01:08:24.931 INFO:teuthology.orchestra.run.smithi120.stdout:1343 punch from 0xc141914 to 0xc143aef, (0x21db bytes) 2024-09-18T01:08:24.931 INFO:teuthology.orchestra.run.smithi120.stdout:1344 read 0xa1911b thru 0xa2590e (0xc7f4 bytes) 2024-09-18T01:08:24.931 INFO:teuthology.orchestra.run.smithi120.stdout:1345 trunc from 0xdead866 to 0xdf6c80a 2024-09-18T01:08:24.937 INFO:teuthology.orchestra.run.smithi120.stdout:1346 writesame 0x7f48084 thru 0x7f53667 data_size 0xe(0xb5e4 bytes) 2024-09-18T01:08:24.945 INFO:teuthology.orchestra.run.smithi120.stdout:1347 trunc from 0xdf6c80a to 0xb34cbbe 2024-09-18T01:08:24.953 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:24.952+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:24.956 INFO:teuthology.orchestra.run.smithi120.stdout:1350 trunc from 0xb34cbbe to 0xa97075a 2024-09-18T01:08:24.965 INFO:teuthology.orchestra.run.smithi120.stdout:1351 punch from 0x224e4b9 to 0x225604b, (0x7b92 bytes) 2024-09-18T01:08:24.966 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:24.967 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:24.971 INFO:teuthology.orchestra.run.smithi120.stdout:1352 trunc from 0xa97075a to 0x5ceabed 2024-09-18T01:08:24.976 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:24.985 INFO:teuthology.orchestra.run.smithi120.stdout:1354 read 0x57583c3 thru 0x575b925 (0x3563 bytes) 2024-09-18T01:08:24.985 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.986 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:24.986 INFO:teuthology.orchestra.run.smithi120.stdout:1357 write 0xf38455 thru 0xf3f566 (0x7112 bytes) 2024-09-18T01:08:24.995 INFO:teuthology.orchestra.run.smithi120.stdout:1359 write 0x6630d78 thru 0x663a5d3 (0x985c bytes) 2024-09-18T01:08:24.998 INFO:teuthology.orchestra.run.smithi120.stdout:1360 read 0x6e16ba thru 0x6e3b40 (0x2487 bytes) 2024-09-18T01:08:24.999 INFO:teuthology.orchestra.run.smithi120.stdout:1361 write 0x5434bf6 thru 0x54358ce (0xcd9 bytes) 2024-09-18T01:08:25.000 INFO:teuthology.orchestra.run.smithi120.stdout:1362 write 0x2d0df68 thru 0x2d1d8e2 (0xf97b bytes) 2024-09-18T01:08:25.002 INFO:teuthology.orchestra.run.smithi120.stdout:1363 trunc from 0x663a5d4 to 0x6640b78 2024-09-18T01:08:25.013 INFO:teuthology.orchestra.run.smithi120.stdout:1364 write 0x6738d93 thru 0x673e20c (0x547a bytes) 2024-09-18T01:08:25.015 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.021 INFO:teuthology.orchestra.run.smithi120.stdout:1366 write 0x78bd18b thru 0x78c5fe8 (0x8e5e bytes) 2024-09-18T01:08:25.023 INFO:teuthology.orchestra.run.smithi120.stdout:1368 punch from 0x2962469 to 0x29640cb, (0x1c62 bytes) 2024-09-18T01:08:25.023 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.023 INFO:teuthology.orchestra.run.smithi120.stdout:1372 punch from 0x3c3d2c4 to 0x3c4c43d, (0xf179 bytes) 2024-09-18T01:08:25.023 INFO:teuthology.orchestra.run.smithi120.stdout:1373 write 0x77d2974 thru 0x77d2c8b (0x318 bytes) 2024-09-18T01:08:25.026 INFO:teuthology.orchestra.run.smithi120.stdout:1374 punch from 0x707046e to 0x7075659, (0x51eb bytes) 2024-09-18T01:08:25.033 INFO:teuthology.orchestra.run.smithi120.stdout:1375 write 0x7ee0859 thru 0x7ee6f4a (0x66f2 bytes) 2024-09-18T01:08:25.034 INFO:teuthology.orchestra.run.smithi120.stdout:1376 read 0x4a30edb thru 0x4a3fe75 (0xef9b bytes) 2024-09-18T01:08:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:1377 writesame 0xaff6fdf thru 0xb003a16 data_size 0x24(0xca38 bytes) 2024-09-18T01:08:25.044 INFO:teuthology.orchestra.run.smithi120.stdout:1378 punch from 0x84e110f to 0x84ebacb, (0xa9bc bytes) 2024-09-18T01:08:25.050 INFO:teuthology.orchestra.run.smithi120.stdout:1379 write 0xc804462 thru 0xc809929 (0x54c8 bytes) 2024-09-18T01:08:25.053 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:25.052+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:25.054 INFO:teuthology.orchestra.run.smithi120.stdout:1380 read 0x5781c3e thru 0x578b597 (0x995a bytes) 2024-09-18T01:08:25.054 INFO:teuthology.orchestra.run.smithi120.stdout:1382 write 0xc67ab33 thru 0xc67f89a (0x4d68 bytes) 2024-09-18T01:08:25.057 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.058 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.058 INFO:teuthology.orchestra.run.smithi120.stdout:1387 read 0x92a6bfd thru 0x92ab016 (0x441a bytes) 2024-09-18T01:08:25.058 INFO:teuthology.orchestra.run.smithi120.stdout:1388 write 0x6a2608e thru 0x6a28a3b (0x29ae bytes) 2024-09-18T01:08:25.059 INFO:teuthology.orchestra.run.smithi120.stdout:1389 write 0x3be3b72 thru 0x3beb413 (0x78a2 bytes) 2024-09-18T01:08:25.061 INFO:teuthology.orchestra.run.smithi120.stdout:1390 read 0x23963b4 thru 0x23a1b47 (0xb794 bytes) 2024-09-18T01:08:25.062 INFO:teuthology.orchestra.run.smithi120.stdout:1391 punch from 0xb554ece to 0xb561f6d, (0xd09f bytes) 2024-09-18T01:08:25.062 INFO:teuthology.orchestra.run.smithi120.stdout:1393 write 0xa1b8bd thru 0xa2b835 (0xff79 bytes) 2024-09-18T01:08:25.063 INFO:teuthology.orchestra.run.smithi120.stdout:1394 read 0x49857fd thru 0x4989379 (0x3b7d bytes) 2024-09-18T01:08:25.063 INFO:teuthology.orchestra.run.smithi120.stdout:1395 write 0x3724f1 thru 0x374d22 (0x2832 bytes) 2024-09-18T01:08:25.064 INFO:teuthology.orchestra.run.smithi120.stdout:1397 write 0x1794778 thru 0x1799056 (0x48df bytes) 2024-09-18T01:08:25.066 INFO:teuthology.orchestra.run.smithi120.stdout:1399 read 0x9a23b14 thru 0x9a29e73 (0x6360 bytes) 2024-09-18T01:08:25.066 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.066 INFO:teuthology.orchestra.run.smithi120.stdout:1402 read 0x15235a3 thru 0x152abe7 (0x7645 bytes) 2024-09-18T01:08:25.067 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.067 INFO:teuthology.orchestra.run.smithi120.stdout:1405 writesame 0x665a55d thru 0x665cb70 data_size 0x2(0x2614 bytes) 2024-09-18T01:08:25.069 INFO:teuthology.orchestra.run.smithi120.stdout:1406 trunc from 0xc80992a to 0xd1667f6 2024-09-18T01:08:25.074 INFO:teuthology.orchestra.run.smithi120.stdout:1407 read 0x2e8e896 thru 0x2e96610 (0x7d7b bytes) 2024-09-18T01:08:25.075 INFO:teuthology.orchestra.run.smithi120.stdout:1408 write 0x8895b09 thru 0x889d384 (0x787c bytes) 2024-09-18T01:08:25.077 INFO:teuthology.orchestra.run.smithi120.stdout:1409 trunc from 0xd1667f6 to 0xd8a53cf 2024-09-18T01:08:25.087 INFO:teuthology.orchestra.run.smithi120.stdout:1410 write 0xde1ffd8 thru 0xde2c3cc (0xc3f5 bytes) 2024-09-18T01:08:25.107 INFO:teuthology.orchestra.run.smithi120.stdout:1411 trunc from 0xde2c3cd to 0x1e25c3 2024-09-18T01:08:25.135 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.051057014999998, 2024-09-18T01:08:25.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:25.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.947742718999997, 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:25.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:25.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:25.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:25.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:25.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:25.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.471066166, 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:25.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.387133927999997, 2024-09-18T01:08:25.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.161 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:25.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:25.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:25.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:25.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.385124795000003, 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:25.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13185 3.a 3:51a12e8c:::rbd_data.117611b60058.00000000000000ee:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.984421245, 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00605274, 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13185 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:25.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148172+0000", 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148174+0000", 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:08:25.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148179+0000", 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148182+0000", 2024-09-18T01:08:25.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2540000000000001e-06 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148734+0000", 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055139499999999999 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.148870+0000", 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013613699999999999 2024-09-18T01:08:25.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.154225+0000", 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053546829999999998 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:13241 3.10 3:090d2a4d:::rbd_data.117611b60058.0000000000000149:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:25.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.960698573999998, 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005514878, 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13241 2024-09-18T01:08:25.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171895+0000", 2024-09-18T01:08:25.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171896+0000", 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171908+0000", 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2401e-05 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171918+0000", 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.042e-05 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171925+0000", 2024-09-18T01:08:25.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.171967+0000", 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1257000000000002e-05 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.172085+0000", 2024-09-18T01:08:25.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011883999999999999 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:45.177409+0000", 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053241210000000002 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:15123 3.1e e43/23 3:7a2bedf1:::rbd_data.1176492bc9ca.0000000000000000:head v 43'60, mlcod=42'58)", 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.567714735000003, 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061319520000000004, 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564878+0000", 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.564879+0000", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565612+0000", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073299200000000002 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565627+0000", 2024-09-18T01:08:25.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5634e-05 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565638+0000", 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0462000000000001e-05 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565707+0000", 2024-09-18T01:08:25.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9402999999999999e-05 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.565730+0000", 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3108999999999998e-05 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.570965+0000", 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052347380000000001 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:49.571010+0000", 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5187e-05 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:25.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.15390107, 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:25.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:25.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18543 3.a 3:5239904c:::rbd_data.11762d95e74a.0000000000000038:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:25.202 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.093071929000001, 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059139489999999999, 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.203 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18543 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039521+0000", 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039523+0000", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.325e-06 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039529+0000", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039541+0000", 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2342e-05 2024-09-18T01:08:25.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039548+0000", 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.99e-06 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039592+0000", 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3915000000000002e-05 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.039646+0000", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3668000000000002e-05 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.040432+0000", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000786288 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.044920+0000", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004487624 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045325+0000", 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000405308 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045381+0000", 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5550000000000002e-05 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:54.045435+0000", 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4657000000000002e-05 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.208 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27452 3.a 3:52657a03:::rbd_data.1176492bc9ca.0000000000000045:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.233915330999999, 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057803079999999996, 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27452 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898678+0000", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898679+0000", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898682+0000", 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8060000000000001e-06 2024-09-18T01:08:25.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898694+0000", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1652e-05 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898700+0000", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4969999999999998e-06 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898807+0000", 2024-09-18T01:08:25.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010601 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898813+0000", 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904378+0000", 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055656710000000003 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904458+0000", 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9935e-05 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27469 3.a 3:52bb01ff:::rbd_data.1176492bc9ca.0000000000000056:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:25.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.233874044, 2024-09-18T01:08:25.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058442279999999999, 2024-09-18T01:08:25.215 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.215 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27469 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898719+0000", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898720+0000", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.37e-07 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898723+0000", 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898730+0000", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898735+0000", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898843+0000", 2024-09-18T01:08:25.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010823900000000001 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.898848+0000", 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8230000000000002e-06 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904484+0000", 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056357589999999997 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904563+0000", 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9746000000000005e-05 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27546 3.a 3:5585dcf7:::rbd_data.1176492bc9ca.00000000000000a3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.233369661000001, 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0055778030000000001, 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27546 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899224+0000", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899226+0000", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5000000000000002e-06 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899234+0000", 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1679999999999998e-06 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899239+0000", 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999998e-06 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899338+0000", 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.912e-05 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.899344+0000", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904713+0000", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053690400000000003 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:56.904801+0000", 2024-09-18T01:08:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7967999999999995e-05 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27837 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.983157305999999, 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056104429999999997, 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27837 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149436+0000", 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149437+0000", 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0999999999999997e-07 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149449+0000", 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2368000000000001e-05 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149462+0000", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3277e-05 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149471+0000", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3799999999999994e-06 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149517+0000", 2024-09-18T01:08:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5838999999999999e-05 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149607+0000", 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0650999999999997e-05 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.149715+0000", 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107345 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154757+0000", 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005042571 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.154948+0000", 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019090200000000001 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155005+0000", 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6504e-05 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.155046+0000", 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1696000000000002e-05 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28147 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.735617872999999, 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006127146, 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28147 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396975+0000", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396977+0000", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4050000000000001e-06 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.396988+0000", 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1046000000000001e-05 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397002+0000", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3895e-05 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397008+0000", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397064+0000", 2024-09-18T01:08:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5961999999999997e-05 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397147+0000", 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3233999999999996e-05 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.397295+0000", 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014745700000000001 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.400032+0000", 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027374460000000001 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403019+0000", 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029870600000000001 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403061+0000", 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1999000000000002e-05 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.403102+0000", 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1072999999999998e-05 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.626830143999999, 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.626802362999999, 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.62538327, 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.497321078999999, 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:40414 3.a e63/51 3:55d9e024:::rbd_data.11762bc2f01c.000000000000004f:head v 63'84, mlcod=63'82)", 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.305351521, 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060175339999999997, 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827242+0000", 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827243+0000", 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.372e-06 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827327+0000", 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3904000000000002e-05 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827346+0000", 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9048000000000001e-05 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827355+0000", 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9129999999999992e-06 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827438+0000", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2930999999999996e-05 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.827465+0000", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7376999999999999e-05 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.833209+0000", 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057439659999999997 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.833259+0000", 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0022999999999999e-05 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.264 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:25.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:25.267 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:25.264+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:1414 trunc from 0x1e25c3 to 0xc5c72e2 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:1415 read 0xafb7143 thru 0xafbdd28 (0x6be6 bytes) 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:1417 write 0xdb0c11c thru 0xdb1a367 (0xe24c bytes) 2024-09-18T01:08:25.268 INFO:teuthology.orchestra.run.smithi120.stdout:1418 trunc from 0xdb1a368 to 0x49ff17c 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1419 read 0x4273fdc thru 0x427f3f5 (0xb41a bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1420 trunc from 0x49ff17c to 0x9dba3f0 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1422 read 0x21e2436 thru 0x21e8e81 (0x6a4c bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1423 read 0x73fe9b8 thru 0x7402175 (0x37be bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1424 read 0x7537efb thru 0x75454cf (0xd5d5 bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1425 trunc from 0x9dba3f0 to 0x24598e8 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1430 writesame 0x13ca522 thru 0x13d6cfb data_size 0x3(0xc7da bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1431 read 0x4177e9 thru 0x41b779 (0x3f91 bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1433 writesame 0x11ad4a3 thru 0x11ba1f9 data_size 0x1(0xcd57 bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1434 read 0x162000a thru 0x162108b (0x1082 bytes) 2024-09-18T01:08:25.269 INFO:teuthology.orchestra.run.smithi120.stdout:1436 read 0x15417db thru 0x1548fa7 (0x77cd bytes) 2024-09-18T01:08:25.270 INFO:teuthology.orchestra.run.smithi120.stdout:1437 read 0x799b69 thru 0x79a45a (0x8f2 bytes) 2024-09-18T01:08:25.270 INFO:teuthology.orchestra.run.smithi120.stdout:1438 punch from 0x1de57d2 to 0x1df36b8, (0xdee6 bytes) 2024-09-18T01:08:25.270 INFO:teuthology.orchestra.run.smithi120.stdout:1439 trunc from 0x24598e8 to 0x7926419 2024-09-18T01:08:25.275 INFO:teuthology.orchestra.run.smithi120.stdout:1441 write 0xaa26b8a thru 0xaa33e8f (0xd306 bytes) 2024-09-18T01:08:25.279 INFO:teuthology.orchestra.run.smithi120.stdout:1443 read 0x3c4cff4 thru 0x3c58baf (0xbbbc bytes) 2024-09-18T01:08:25.288 INFO:teuthology.orchestra.run.smithi120.stdout:1446 trunc from 0xaa33e90 to 0x3841da1 2024-09-18T01:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout:1447 punch from 0x2bfbd97 to 0x2c0aad3, (0xed3c bytes) 2024-09-18T01:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout:1450 read 0x354ca73 thru 0x35581a4 (0xb732 bytes) 2024-09-18T01:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout:1454 read 0x232be73 thru 0x232d071 (0x11ff bytes) 2024-09-18T01:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout:1455 write 0x153aded thru 0x1547117 (0xc32b bytes) 2024-09-18T01:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout:1456 trunc from 0x3841da1 to 0x47139ab 2024-09-18T01:08:25.311 INFO:teuthology.orchestra.run.smithi120.stdout:1457 write 0x2aeec23 thru 0x2afa8a0 (0xbc7e bytes) 2024-09-18T01:08:25.329 INFO:teuthology.orchestra.run.smithi120.stdout:1458 write 0xca1b8ec thru 0xca1d2e0 (0x19f5 bytes) 2024-09-18T01:08:25.332 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.332 INFO:teuthology.orchestra.run.smithi120.stdout:1460 read 0xa0528ae thru 0xa0627c7 (0xff1a bytes) 2024-09-18T01:08:25.332 INFO:teuthology.orchestra.run.smithi120.stdout:1461 writesame 0x1bc13e6 thru 0x1bcc433 data_size 0x1(0xb04e bytes) 2024-09-18T01:08:25.356 INFO:teuthology.orchestra.run.smithi120.stdout:1463 read 0xb0b0d1c thru 0xb0bd8d6 (0xcbbb bytes) 2024-09-18T01:08:25.356 INFO:teuthology.orchestra.run.smithi120.stdout:1464 write 0x70048d2 thru 0x700ac62 (0x6391 bytes) 2024-09-18T01:08:25.361 INFO:teuthology.orchestra.run.smithi120.stdout:1465 read 0x4f5b0d5 thru 0x4f5e941 (0x386d bytes) 2024-09-18T01:08:25.363 INFO:teuthology.orchestra.run.smithi120.stdout:1466 trunc from 0xca1d2e1 to 0xac79fed 2024-09-18T01:08:25.366 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:25.364+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:25.370 INFO:teuthology.orchestra.run.smithi120.stdout:1468 read 0x638282 thru 0x639334 (0x10b3 bytes) 2024-09-18T01:08:25.370 INFO:teuthology.orchestra.run.smithi120.stdout:1470 punch from 0xa992b7 to 0xaa6890, (0xd5d9 bytes) 2024-09-18T01:08:25.376 INFO:teuthology.orchestra.run.smithi120.stdout:1471 write 0xad17d25 thru 0xad1bfc1 (0x429d bytes) 2024-09-18T01:08:25.380 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.380 INFO:teuthology.orchestra.run.smithi120.stdout:1475 write 0x6f436b2 thru 0x6f491d1 (0x5b20 bytes) 2024-09-18T01:08:25.383 INFO:teuthology.orchestra.run.smithi120.stdout:1476 punch from 0x1b0e730 to 0x1b1a3de, (0xbcae bytes) 2024-09-18T01:08:25.383 INFO:teuthology.orchestra.run.smithi120.stdout:1477 write 0x759dd86 thru 0x75ad4df (0xf75a bytes) 2024-09-18T01:08:25.387 INFO:teuthology.orchestra.run.smithi120.stdout:1478 write 0x6fa29c7 thru 0x6faea69 (0xc0a3 bytes) 2024-09-18T01:08:25.389 INFO:teuthology.orchestra.run.smithi120.stdout:1480 write 0xa662629 thru 0xa66fba0 (0xd578 bytes) 2024-09-18T01:08:25.393 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.393 INFO:teuthology.orchestra.run.smithi120.stdout:1482 writesame 0x12696e9 thru 0x1276dae data_size 0x25(0xd6c6 bytes) 2024-09-18T01:08:25.395 INFO:teuthology.orchestra.run.smithi120.stdout:1483 writesame 0x1c1a34d thru 0x1c2235b data_size 0x1(0x800f bytes) 2024-09-18T01:08:25.411 INFO:teuthology.orchestra.run.smithi120.stdout:1484 write 0x1ada302 thru 0x1ae0af8 (0x67f7 bytes) 2024-09-18T01:08:25.412 INFO:teuthology.orchestra.run.smithi120.stdout:1485 writesame 0x848b9b1 thru 0x8495e79 data_size 0xd(0xa4c9 bytes) 2024-09-18T01:08:25.415 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:25.415 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:25.416 INFO:teuthology.orchestra.run.smithi120.stdout:1487 write 0xa558850 thru 0xa563542 (0xacf3 bytes) 2024-09-18T01:08:25.418 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.418 INFO:teuthology.orchestra.run.smithi120.stdout:1490 punch from 0x1d07404 to 0x1d10872, (0x946e bytes) 2024-09-18T01:08:25.418 INFO:teuthology.orchestra.run.smithi120.stdout:1491 punch from 0x521ed68 to 0x522ea82, (0xfd1a bytes) 2024-09-18T01:08:25.419 INFO:teuthology.orchestra.run.smithi120.stdout:1492 writesame 0x400ee31 thru 0x401c02e data_size 0x1(0xd1fe bytes) 2024-09-18T01:08:25.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:25.445 INFO:teuthology.orchestra.run.smithi120.stdout:1493 read 0x1535f88 thru 0x153de8f (0x7f08 bytes) 2024-09-18T01:08:25.447 INFO:teuthology.orchestra.run.smithi120.stdout:1495 trunc from 0xad1bfc2 to 0xbcd533a 2024-09-18T01:08:25.452 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.452 INFO:teuthology.orchestra.run.smithi120.stdout:1498 punch from 0x49dee39 to 0x49eb1c3, (0xc38a bytes) 2024-09-18T01:08:25.452 INFO:teuthology.orchestra.run.smithi120.stdout:1500 punch from 0x9f95721 to 0x9fa384b, (0xe12a bytes) 2024-09-18T01:08:25.452 INFO:teuthology.orchestra.run.smithi120.stdout:1501 write 0x4d00014 thru 0x4d0b3dd (0xb3ca bytes) 2024-09-18T01:08:25.460 INFO:teuthology.orchestra.run.smithi120.stdout:1502 trunc from 0xbcd533a to 0x6639557 2024-09-18T01:08:25.468 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:25.467+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:25.483 INFO:teuthology.orchestra.run.smithi120.stdout:1503 write 0x5a8914c thru 0x5a96325 (0xd1da bytes) 2024-09-18T01:08:25.495 INFO:teuthology.orchestra.run.smithi120.stdout:1505 trunc from 0x6639557 to 0xdb9d3e4 2024-09-18T01:08:25.513 INFO:teuthology.orchestra.run.smithi120.stdout:1506 trunc from 0xdb9d3e4 to 0x5507814 2024-09-18T01:08:25.523 INFO:teuthology.orchestra.run.smithi120.stdout:1507 writesame 0x102dd66 thru 0x103ca3c data_size 0x1(0xecd7 bytes) 2024-09-18T01:08:25.533 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:25.533 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:25.543 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:25.551 INFO:teuthology.orchestra.run.smithi120.stdout:1508 punch from 0x2bd8da1 to 0x2be03d3, (0x7632 bytes) 2024-09-18T01:08:25.562 INFO:teuthology.orchestra.run.smithi120.stdout:1509 write 0x739789f thru 0x73978a5 (0x7 bytes) 2024-09-18T01:08:25.563 INFO:teuthology.orchestra.run.smithi120.stdout:1510 punch from 0x48a5b77 to 0x48add8e, (0x8217 bytes) 2024-09-18T01:08:25.577 INFO:teuthology.orchestra.run.smithi120.stdout:1511 writesame 0xe1c6ed9 thru 0xe1d4934 data_size 0x64(0xda5c bytes) 2024-09-18T01:08:25.577 INFO:teuthology.orchestra.run.smithi120.stdout:1512 write 0xc1c99c2 thru 0xc1cc418 (0x2a57 bytes) 2024-09-18T01:08:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:1513 write 0x16bb771 thru 0x16cb6f9 (0xff89 bytes) 2024-09-18T01:08:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:1518 read 0x642780a thru 0x643112e (0x9925 bytes) 2024-09-18T01:08:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:1523 punch from 0x4f87152 to 0x4f899a5, (0x2853 bytes) 2024-09-18T01:08:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:1524 read 0x61f012 thru 0x6209c1 (0x19b0 bytes) 2024-09-18T01:08:25.586 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:25.584+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:25.586 INFO:teuthology.orchestra.run.smithi120.stdout:1525 trunc from 0xe1d4935 to 0xcfdf78b 2024-09-18T01:08:25.601 INFO:teuthology.orchestra.run.smithi120.stdout:1527 trunc from 0xcfdf78b to 0x476b2ac 2024-09-18T01:08:25.633 INFO:teuthology.orchestra.run.smithi120.stdout:1530 writesame 0xd40039a thru 0xd4066b5 data_size 0x2(0x631c bytes) 2024-09-18T01:08:25.633 INFO:teuthology.orchestra.run.smithi120.stdout:1531 clone 17 order 22 su 524288 sc 9 2024-09-18T01:08:25.668 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:25.667+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.792206449000005, 2024-09-18T01:08:25.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:25.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:25.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:25.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.399027150999999, 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:25.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:25.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:25.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.032295851000001, 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:25.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.752908701999999, 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.060586978000003, 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.060571881000001, 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.060560731000002, 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.060551093999997, 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.056639046000001, 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.993137795999999, 2024-09-18T01:08:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43586 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5816320~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295155078000001, 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015218535, 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43586 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2499999999999999e-07 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394861+0000", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394864+0000", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394867+0000", 2024-09-18T01:08:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.409942+0000", 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015075312 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410015+0000", 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3090000000000007e-05 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410079+0000", 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3126999999999994e-05 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43587 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5980160~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295145212, 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015359998, 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43587 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394871+0000", 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4200000000000004e-07 2024-09-18T01:08:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394874+0000", 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394876+0000", 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4399999999999999e-06 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410105+0000", 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015228759 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410155+0000", 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9987000000000001e-05 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410230+0000", 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4697999999999995e-05 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43588 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6144000~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295135847999999, 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015489108999999999, 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43588 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394880+0000", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394884+0000", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6829999999999999e-06 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394888+0000", 2024-09-18T01:08:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3409999999999999e-06 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394890+0000", 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.543e-06 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410258+0000", 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015368371 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410303+0000", 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410368+0000", 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5747999999999995e-05 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43589 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6307840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.29512128, 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015665806000000001, 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43589 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:08:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394895+0000", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0079999999999999e-06 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394904+0000", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.234e-06 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394909+0000", 2024-09-18T01:08:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410406+0000", 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015497273000000001 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410486+0000", 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9722999999999995e-05 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410560+0000", 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4008e-05 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43590 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6471680~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295102198, 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01587189, 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43590 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8500000000000002e-07 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394915+0000", 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3239999999999999e-06 2024-09-18T01:08:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394918+0000", 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8789999999999996e-06 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394921+0000", 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410601+0000", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015679648000000001 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410683+0000", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2826999999999998e-05 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410785+0000", 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101365 2024-09-18T01:08:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43591 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6635520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295090974000001, 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015982698, 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43591 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394925+0000", 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.006e-06 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394928+0000", 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394931+0000", 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5720000000000001e-06 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410803+0000", 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015871962999999999 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410847+0000", 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3912000000000002e-05 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410907+0000", 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0025999999999998e-05 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43592 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6799360~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295066617, 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016170249000000001, 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43592 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394950+0000", 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.161e-06 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394955+0000", 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9089999999999999e-06 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394958+0000", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.593e-06 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410941+0000", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015983431999999999 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411019+0000", 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8325999999999995e-05 2024-09-18T01:08:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411119+0000", 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9463999999999995e-05 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50421 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.45707454600000003, 2024-09-18T01:08:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020565198, 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50421 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232946+0000", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3209999999999998e-06 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232956+0000", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0368e-05 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232963+0000", 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2400000000000001e-06 2024-09-18T01:08:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233001+0000", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7298000000000001e-05 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233052+0000", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1659000000000001e-05 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.236268+0000", 2024-09-18T01:08:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003215894 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253414+0000", 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017146122999999999 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253429+0000", 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4674e-05 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253468+0000", 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.896e-05 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253506+0000", 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7793999999999998e-05 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50514 3.11 3:8aff915a:::rbd_data.11761711706f.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.294224074, 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015404816999999999, 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50514 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395792+0000", 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6300000000000005e-07 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395796+0000", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4490000000000001e-06 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395805+0000", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8580000000000006e-06 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395810+0000", 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395844+0000", 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769000000000001e-05 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395895+0000", 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0930000000000002e-05 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.398279+0000", 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023842569999999999 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411026+0000", 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012747 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411078+0000", 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2296000000000002e-05 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411143+0000", 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4561999999999996e-05 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411196+0000", 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3152999999999997e-05 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50582 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.16624677800000001, 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027666276, 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50582 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523769+0000", 2024-09-18T01:08:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8599999999999997e-07 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523774+0000", 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523783+0000", 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5149999999999995e-06 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523788+0000", 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523838+0000", 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9438999999999999e-05 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523930+0000", 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1978000000000006e-05 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.528122+0000", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004192749 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551279+0000", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023157015999999999 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551339+0000", 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9410000000000002e-05 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551380+0000", 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0927000000000003e-05 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551435+0000", 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4846999999999999e-05 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:25.804 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:08:25.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:08:25.807 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:25.804+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:25.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:25.906 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:25.904+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:25.954 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:25.954 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:25.954 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:25.954 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:25.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:26.008 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:26.005+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:26.108 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:26.106+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:26.109 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:26.109 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:26.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:26.209 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:26.208+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.673412263000003, 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:26.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:26.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:26.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.66977851, 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.669754941999997, 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.668179539999997, 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.663746449000001, 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.662424272000003, 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.658370460999997, 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.632682031999998, 2024-09-18T01:08:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.602143826999999, 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.601654134999997, 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.537294439, 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366256806999999, 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366228040999999, 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366204794, 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366184791, 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366166148, 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.366147606, 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.36612702, 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.36608275, 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.365102558, 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:26.376 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:26.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:26.378 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:26.377+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:26.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:26.478 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:26.476+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:26.524 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:26.525 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:26.525 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:26.525 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:26.534 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:26.539 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:26.539 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:26.556 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:26.557 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:26.573 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:26.573 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:26.579 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:26.578+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:26.589 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:26.606 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:26.606 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:26.623 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:26.623 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:26.640 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:26.640 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:26.643 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:26.643 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:26.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:26.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:08:26.657 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:26.680 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:26.679+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:26.764 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:26.764 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:26.775 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:08:26.781 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:26.780+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:26.796 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:08:26.882 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:26.880+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:26.902 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:26.902 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:26.911 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:26.983 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:26.982+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:27.056 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.056 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.065 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:27.109 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:27.107+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:27.169 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.169 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.178 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:27.185 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:27.184+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:27.286 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:27.285+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:27.320 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.321 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.329 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:27.388 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:27.387+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:27.431 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.431 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:27.489 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:27.488+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:27.543 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.543 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.551 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:27.589 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:27.588+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:27.667 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.668 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:27.720 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:27.719+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:27.783 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.783 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.783 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:08:27.784 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:08:27.791 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:27.790+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:27.792 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:27.801 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:08:27.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:08:27.892 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:27.890+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:27.934 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-18T01:08:27.938 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:27.938 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:27.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:27.994 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:27.992+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:28.053 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.054 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:28.094 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:28.092+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:28.163 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:08:28.168 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.169 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.177 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-18T01:08:27.778696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":678120542,"num_objects":696,"num_object_clones":0,"num_object_copies":1392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":696,"num_whiteouts":0,"num_read":46887,"num_read_kb":645888,"num_write":9283,"num_write_kb":406678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5869,"ondisk_log_size":5869,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":735480,"kb_used_data":465160,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754239240,"statfs":{"total":773094113280,"available":772340981760,"internally_reserved":0,"allocated":476323840,"data_stored":470726265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70848002,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1992,"num_read_kb":6477,"num_write":382,"num_write_kb":5216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021912"},"pg_stats":[{"pgid":"3.1f","version":"77'191","reported_seq":1577,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783725+0000","last_change":"2024-09-18T01:08:24.552111+0000","last_active":"2024-09-18T01:08:26.783725+0000","last_peered":"2024-09-18T01:08:26.783725+0000","last_clean":"2024-09-18T01:08:26.783725+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:08:26.783725+0000","last_undegraded":"2024-09-18T01:08:26.783725+0000","last_fullsized":"2024-09-18T01:08:26.783725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:53:24.429489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010483669999999999,"stat_sum":{"num_bytes":20862296,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1758,"num_read_kb":19496,"num_write":226,"num_write_kb":17312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1e","version":"74'90","reported_seq":724,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.546374+0000","last_change":"2024-09-18T01:08:24.546569+0000","last_active":"2024-09-18T01:08:24.546374+0000","last_peered":"2024-09-18T01:08:24.546374+0000","last_clean":"2024-09-18T01:08:24.546374+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:08:24.546374+0000","last_undegraded":"2024-09-18T01:08:24.546374+0000","last_fullsized":"2024-09-18T01:08:24.546374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:27.427417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285209999999999,"stat_sum":{"num_bytes":10656621,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":508,"num_read_kb":16488,"num_write":175,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1d","version":"79'120","reported_seq":1228,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.803490+0000","last_change":"2024-09-18T01:08:24.697488+0000","last_active":"2024-09-18T01:08:26.803490+0000","last_peered":"2024-09-18T01:08:26.803490+0000","last_clean":"2024-09-18T01:08:26.803490+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:08:26.803490+0000","last_undegraded":"2024-09-18T01:08:26.803490+0000","last_fullsized":"2024-09-18T01:08:26.803490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:18.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048755899999999999,"stat_sum":{"num_bytes":30508568,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":870,"num_read_kb":26307,"num_write":211,"num_write_kb":28930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1c","version":"78'105","reported_seq":833,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.841188+0000","last_change":"2024-09-18T01:08:24.697177+0000","last_active":"2024-09-18T01:08:26.841188+0000","last_peered":"2024-09-18T01:08:26.841188+0000","last_clean":"2024-09-18T01:08:26.841188+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:08:26.841188+0000","last_undegraded":"2024-09-18T01:08:26.841188+0000","last_fullsized":"2024-09-18T01:08:26.841188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:10.636440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046942400000000002,"stat_sum":{"num_bytes":21004077,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":530,"num_read_kb":21423,"num_write":130,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1b","version":"79'429","reported_seq":2086,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.831977+0000","last_change":"2024-09-18T01:08:24.552678+0000","last_active":"2024-09-18T01:08:26.831977+0000","last_peered":"2024-09-18T01:08:26.831977+0000","last_clean":"2024-09-18T01:08:26.831977+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:08:26.831977+0000","last_undegraded":"2024-09-18T01:08:26.831977+0000","last_fullsized":"2024-09-18T01:08:26.831977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:44.646917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012655590000000001,"stat_sum":{"num_bytes":24953907,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3261,"num_read_kb":26179,"num_write":704,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1a","version":"74'109","reported_seq":1364,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:25.616448+0000","last_change":"2024-09-18T01:08:24.548295+0000","last_active":"2024-09-18T01:08:25.616448+0000","last_peered":"2024-09-18T01:08:25.616448+0000","last_clean":"2024-09-18T01:08:25.616448+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:08:25.616448+0000","last_undegraded":"2024-09-18T01:08:25.616448+0000","last_fullsized":"2024-09-18T01:08:25.616448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:26.875969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000409963,"stat_sum":{"num_bytes":15596178,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1335,"num_read_kb":20763,"num_write":173,"num_write_kb":11453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.19","version":"77'102","reported_seq":1269,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.824786+0000","last_change":"2024-09-18T01:08:24.547399+0000","last_active":"2024-09-18T01:08:26.824786+0000","last_peered":"2024-09-18T01:08:26.824786+0000","last_clean":"2024-09-18T01:08:26.824786+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:08:26.824786+0000","last_undegraded":"2024-09-18T01:08:26.824786+0000","last_fullsized":"2024-09-18T01:08:26.824786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:19:43.484695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052836400000000003,"stat_sum":{"num_bytes":28760688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":954,"num_read_kb":17758,"num_write":137,"num_write_kb":10788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.18","version":"73'124","reported_seq":2246,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545769+0000","last_change":"2024-09-18T01:08:24.545769+0000","last_active":"2024-09-18T01:08:24.545769+0000","last_peered":"2024-09-18T01:08:24.545769+0000","last_clean":"2024-09-18T01:08:24.545769+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:08:24.545769+0000","last_undegraded":"2024-09-18T01:08:24.545769+0000","last_fullsized":"2024-09-18T01:08:24.545769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:29.187547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091264099999999995,"stat_sum":{"num_bytes":29763459,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1964,"num_read_kb":29968,"num_write":175,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.17","version":"73'131","reported_seq":1065,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783784+0000","last_change":"2024-09-18T01:08:24.552789+0000","last_active":"2024-09-18T01:08:26.783784+0000","last_peered":"2024-09-18T01:08:26.783784+0000","last_clean":"2024-09-18T01:08:26.783784+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:08:26.783784+0000","last_undegraded":"2024-09-18T01:08:26.783784+0000","last_fullsized":"2024-09-18T01:08:26.783784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:43.664728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011101629999999999,"stat_sum":{"num_bytes":29119086,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":893,"num_read_kb":20345,"num_write":228,"num_write_kb":11682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.16","version":"71'86","reported_seq":1054,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788711+0000","last_change":"2024-09-18T01:08:24.697170+0000","last_active":"2024-09-18T01:08:26.788711+0000","last_peered":"2024-09-18T01:08:26.788711+0000","last_clean":"2024-09-18T01:08:26.788711+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:08:26.788711+0000","last_undegraded":"2024-09-18T01:08:26.788711+0000","last_fullsized":"2024-09-18T01:08:26.788711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:14.837855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050167300000000001,"stat_sum":{"num_bytes":25668979,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":762,"num_read_kb":14832,"num_write":124,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.15","version":"79'146","reported_seq":2232,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788670+0000","last_change":"2024-09-18T01:08:24.557082+0000","last_active":"2024-09-18T01:08:26.788670+0000","last_peered":"2024-09-18T01:08:26.788670+0000","last_clean":"2024-09-18T01:08:26.788670+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:08:26.788670+0000","last_undegraded":"2024-09-18T01:08:26.788670+0000","last_fullsized":"2024-09-18T01:08:26.788670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:12.588382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057264499999999999,"stat_sum":{"num_bytes":30703508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2078,"num_read_kb":39372,"num_write":170,"num_write_kb":17302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.14","version":"74'325","reported_seq":881,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.547759+0000","last_change":"2024-09-18T01:08:24.547945+0000","last_active":"2024-09-18T01:08:24.547759+0000","last_peered":"2024-09-18T01:08:24.547759+0000","last_clean":"2024-09-18T01:08:24.547759+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:08:24.547759+0000","last_undegraded":"2024-09-18T01:08:24.547759+0000","last_fullsized":"2024-09-18T01:08:24.547759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:38.320977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082267699999999996,"stat_sum":{"num_bytes":6357140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1616,"num_read_kb":11155,"num_write":529,"num_write_kb":5563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788767+0000","last_change":"2024-09-18T01:07:58.635173+0000","last_active":"2024-09-18T01:08:26.788767+0000","last_peered":"2024-09-18T01:08:26.788767+0000","last_clean":"2024-09-18T01:08:26.788767+0000","last_became_active":"2024-09-18T01:07:58.634732+0000","last_became_peered":"2024-09-18T01:07:58.634732+0000","last_unstale":"2024-09-18T01:08:26.788767+0000","last_undegraded":"2024-09-18T01:08:26.788767+0000","last_fullsized":"2024-09-18T01:08:26.788767+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:52:17.327995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"74'139","reported_seq":1410,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783559+0000","last_change":"2024-09-18T01:08:24.552000+0000","last_active":"2024-09-18T01:08:26.783559+0000","last_peered":"2024-09-18T01:08:26.783559+0000","last_clean":"2024-09-18T01:08:26.783559+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:08:26.783559+0000","last_undegraded":"2024-09-18T01:08:26.783559+0000","last_fullsized":"2024-09-18T01:08:26.783559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:39:08.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093537100000000005,"stat_sum":{"num_bytes":10101919,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1853,"num_read_kb":14920,"num_write":236,"num_write_kb":9399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.778146+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:08:26.778146+0000","last_peered":"2024-09-18T01:08:26.778146+0000","last_clean":"2024-09-18T01:08:26.778146+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:08:26.778146+0000","last_undegraded":"2024-09-18T01:08:26.778146+0000","last_fullsized":"2024-09-18T01:08:26.778146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"64'185","reported_seq":846,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545857+0000","last_change":"2024-09-18T01:08:24.546273+0000","last_active":"2024-09-18T01:08:24.545857+0000","last_peered":"2024-09-18T01:08:24.545857+0000","last_clean":"2024-09-18T01:08:24.545857+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:08:24.545857+0000","last_undegraded":"2024-09-18T01:08:24.545857+0000","last_fullsized":"2024-09-18T01:08:24.545857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:03.212528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013889950000000001,"stat_sum":{"num_bytes":9748480,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1031,"num_read_kb":17129,"num_write":349,"num_write_kb":12969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788913+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:08:26.788913+0000","last_peered":"2024-09-18T01:08:26.788913+0000","last_clean":"2024-09-18T01:08:26.788913+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:08:26.788913+0000","last_undegraded":"2024-09-18T01:08:26.788913+0000","last_fullsized":"2024-09-18T01:08:26.788913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'134","reported_seq":1859,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.821066+0000","last_change":"2024-09-18T01:08:24.547682+0000","last_active":"2024-09-18T01:08:26.821066+0000","last_peered":"2024-09-18T01:08:26.821066+0000","last_clean":"2024-09-18T01:08:26.821066+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:08:26.821066+0000","last_undegraded":"2024-09-18T01:08:26.821066+0000","last_fullsized":"2024-09-18T01:08:26.821066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:10.763832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000415905,"stat_sum":{"num_bytes":30193482,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1829,"num_read_kb":28850,"num_write":143,"num_write_kb":9207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.778046+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:08:26.778046+0000","last_peered":"2024-09-18T01:08:26.778046+0000","last_clean":"2024-09-18T01:08:26.778046+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:08:26.778046+0000","last_undegraded":"2024-09-18T01:08:26.778046+0000","last_fullsized":"2024-09-18T01:08:26.778046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"72'256","reported_seq":2324,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788865+0000","last_change":"2024-09-18T01:08:24.697590+0000","last_active":"2024-09-18T01:08:26.788865+0000","last_peered":"2024-09-18T01:08:26.788865+0000","last_clean":"2024-09-18T01:08:26.788865+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:08:26.788865+0000","last_undegraded":"2024-09-18T01:08:26.788865+0000","last_fullsized":"2024-09-18T01:08:26.788865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:48.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056296500000000001,"stat_sum":{"num_bytes":41103835,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2514,"num_read_kb":38721,"num_write":498,"num_write_kb":23286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.3","version":"74'210","reported_seq":1276,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.547823+0000","last_change":"2024-09-18T01:08:24.548053+0000","last_active":"2024-09-18T01:08:24.547823+0000","last_peered":"2024-09-18T01:08:24.547823+0000","last_clean":"2024-09-18T01:08:24.547823+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:08:24.547823+0000","last_undegraded":"2024-09-18T01:08:24.547823+0000","last_fullsized":"2024-09-18T01:08:24.547823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:37.905731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052537600000000003,"stat_sum":{"num_bytes":9438348,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1619,"num_read_kb":15315,"num_write":294,"num_write_kb":7339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":136,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788458+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:08:26.788458+0000","last_peered":"2024-09-18T01:08:26.788458+0000","last_clean":"2024-09-18T01:08:26.788458+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:08:26.788458+0000","last_undegraded":"2024-09-18T01:08:26.788458+0000","last_fullsized":"2024-09-18T01:08:26.788458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"74'136","reported_seq":1799,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.777615+0000","last_change":"2024-09-18T01:08:24.548079+0000","last_active":"2024-09-18T01:08:26.777615+0000","last_peered":"2024-09-18T01:08:26.777615+0000","last_clean":"2024-09-18T01:08:26.777615+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:08:26.777615+0000","last_undegraded":"2024-09-18T01:08:26.777615+0000","last_fullsized":"2024-09-18T01:08:26.777615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:39.957103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078098800000000004,"stat_sum":{"num_bytes":28013935,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1686,"num_read_kb":36481,"num_write":297,"num_write_kb":17730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.809288+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:08:26.809288+0000","last_peered":"2024-09-18T01:08:26.809288+0000","last_clean":"2024-09-18T01:08:26.809288+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:08:26.809288+0000","last_undegraded":"2024-09-18T01:08:26.809288+0000","last_fullsized":"2024-09-18T01:08:26.809288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"77'156","reported_seq":1397,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.825939+0000","last_change":"2024-09-18T01:08:24.551863+0000","last_active":"2024-09-18T01:08:26.825939+0000","last_peered":"2024-09-18T01:08:26.825939+0000","last_clean":"2024-09-18T01:08:26.825939+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:08:26.825939+0000","last_undegraded":"2024-09-18T01:08:26.825939+0000","last_fullsized":"2024-09-18T01:08:26.825939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:48.860383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090600000000004,"stat_sum":{"num_bytes":32397194,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1280,"num_read_kb":13078,"num_write":158,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788305+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:08:26.788305+0000","last_peered":"2024-09-18T01:08:26.788305+0000","last_clean":"2024-09-18T01:08:26.788305+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:08:26.788305+0000","last_undegraded":"2024-09-18T01:08:26.788305+0000","last_fullsized":"2024-09-18T01:08:26.788305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"71'150","reported_seq":1307,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.694637+0000","last_change":"2024-09-18T01:08:24.546252+0000","last_active":"2024-09-18T01:08:24.694637+0000","last_peered":"2024-09-18T01:08:24.694637+0000","last_clean":"2024-09-18T01:08:24.694637+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:08:24.694637+0000","last_undegraded":"2024-09-18T01:08:24.694637+0000","last_fullsized":"2024-09-18T01:08:24.694637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:35:38.245719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011962030000000001,"stat_sum":{"num_bytes":19203626,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1272,"num_read_kb":12774,"num_write":183,"num_write_kb":17754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788499+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:08:26.788499+0000","last_peered":"2024-09-18T01:08:26.788499+0000","last_clean":"2024-09-18T01:08:26.788499+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:08:26.788499+0000","last_undegraded":"2024-09-18T01:08:26.788499+0000","last_fullsized":"2024-09-18T01:08:26.788499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":286,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788379+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:08:26.788379+0000","last_peered":"2024-09-18T01:08:26.788379+0000","last_clean":"2024-09-18T01:08:26.788379+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:08:26.788379+0000","last_undegraded":"2024-09-18T01:08:26.788379+0000","last_fullsized":"2024-09-18T01:08:26.788379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'480","reported_seq":1447,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.546523+0000","last_change":"2024-09-18T01:08:24.546523+0000","last_active":"2024-09-18T01:08:24.546523+0000","last_peered":"2024-09-18T01:08:24.546523+0000","last_clean":"2024-09-18T01:08:24.546523+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:08:24.546523+0000","last_undegraded":"2024-09-18T01:08:24.546523+0000","last_fullsized":"2024-09-18T01:08:24.546523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:04:51.061212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010885619999999999,"stat_sum":{"num_bytes":8273963,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2742,"num_read_kb":12424,"num_write":926,"num_write_kb":5521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.9","version":"77'73","reported_seq":699,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:25.584672+0000","last_change":"2024-09-18T01:08:24.547523+0000","last_active":"2024-09-18T01:08:25.584672+0000","last_peered":"2024-09-18T01:08:25.584672+0000","last_clean":"2024-09-18T01:08:25.584672+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:08:25.584672+0000","last_undegraded":"2024-09-18T01:08:25.584672+0000","last_fullsized":"2024-09-18T01:08:25.584672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:50.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046679600000000001,"stat_sum":{"num_bytes":19645466,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":396,"num_read_kb":19056,"num_write":79,"num_write_kb":8783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.a","version":"71'101","reported_seq":1280,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.548015+0000","last_change":"2024-09-18T01:08:24.548165+0000","last_active":"2024-09-18T01:08:24.548015+0000","last_peered":"2024-09-18T01:08:24.548015+0000","last_clean":"2024-09-18T01:08:24.548015+0000","last_became_active":"2024-09-18T01:07:58.640172+0000","last_became_peered":"2024-09-18T01:07:58.640172+0000","last_unstale":"2024-09-18T01:08:24.548015+0000","last_undegraded":"2024-09-18T01:08:24.548015+0000","last_fullsized":"2024-09-18T01:08:24.548015+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:33.051870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060734799999999996,"stat_sum":{"num_bytes":42510450,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":948,"num_read_kb":16793,"num_write":161,"num_write_kb":7285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.b","version":"74'149","reported_seq":1079,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545428+0000","last_change":"2024-09-18T01:08:24.545761+0000","last_active":"2024-09-18T01:08:24.545428+0000","last_peered":"2024-09-18T01:08:24.545428+0000","last_clean":"2024-09-18T01:08:24.545428+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:08:24.545428+0000","last_undegraded":"2024-09-18T01:08:24.545428+0000","last_fullsized":"2024-09-18T01:08:24.545428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:43.756298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089125599999999999,"stat_sum":{"num_bytes":12508096,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":978,"num_read_kb":18864,"num_write":243,"num_write_kb":8595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.c","version":"79'479","reported_seq":1882,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788595+0000","last_change":"2024-09-18T01:08:24.697483+0000","last_active":"2024-09-18T01:08:26.788595+0000","last_peered":"2024-09-18T01:08:26.788595+0000","last_clean":"2024-09-18T01:08:26.788595+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:08:26.788595+0000","last_undegraded":"2024-09-18T01:08:26.788595+0000","last_fullsized":"2024-09-18T01:08:26.788595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:22:21.343183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000613183,"stat_sum":{"num_bytes":13484728,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":3089,"num_read_kb":17421,"num_write":937,"num_write_kb":7766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.d","version":"71'111","reported_seq":1195,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788792+0000","last_change":"2024-09-18T01:08:24.557303+0000","last_active":"2024-09-18T01:08:26.788792+0000","last_peered":"2024-09-18T01:08:26.788792+0000","last_clean":"2024-09-18T01:08:26.788792+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:08:26.788792+0000","last_undegraded":"2024-09-18T01:08:26.788792+0000","last_fullsized":"2024-09-18T01:08:26.788792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:27:52.623740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082317600000000001,"stat_sum":{"num_bytes":17517309,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1092,"num_read_kb":22631,"num_write":139,"num_write_kb":15375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.e","version":"79'335","reported_seq":2297,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.789354+0000","last_change":"2024-09-18T01:08:24.558122+0000","last_active":"2024-09-18T01:08:26.789354+0000","last_peered":"2024-09-18T01:08:26.789354+0000","last_clean":"2024-09-18T01:08:26.789354+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:08:26.789354+0000","last_undegraded":"2024-09-18T01:08:26.789354+0000","last_fullsized":"2024-09-18T01:08:26.789354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:36.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010028979999999999,"stat_sum":{"num_bytes":33326407,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2722,"num_read_kb":41024,"num_write":554,"num_write_kb":26929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.f","version":"74'135","reported_seq":1062,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788694+0000","last_change":"2024-09-18T01:08:24.557394+0000","last_active":"2024-09-18T01:08:26.788694+0000","last_peered":"2024-09-18T01:08:26.788694+0000","last_clean":"2024-09-18T01:08:26.788694+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:08:26.788694+0000","last_undegraded":"2024-09-18T01:08:26.788694+0000","last_fullsized":"2024-09-18T01:08:26.788694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:45.831846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050360900000000002,"stat_sum":{"num_bytes":9284284,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1007,"num_read_kb":13572,"num_write":220,"num_write_kb":6993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.10","version":"74'110","reported_seq":882,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783635+0000","last_change":"2024-09-18T01:08:24.552239+0000","last_active":"2024-09-18T01:08:26.783635+0000","last_peered":"2024-09-18T01:08:26.783635+0000","last_clean":"2024-09-18T01:08:26.783635+0000","last_became_active":"2024-09-18T01:07:58.634581+0000","last_became_peered":"2024-09-18T01:07:58.634581+0000","last_unstale":"2024-09-18T01:08:26.783635+0000","last_undegraded":"2024-09-18T01:08:26.783635+0000","last_fullsized":"2024-09-18T01:08:26.783635+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:26.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083914099999999995,"stat_sum":{"num_bytes":11778783,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":624,"num_read_kb":13660,"num_write":134,"num_write_kb":10782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.11","version":"77'123","reported_seq":909,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.824247+0000","last_change":"2024-09-18T01:08:24.556742+0000","last_active":"2024-09-18T01:08:26.824247+0000","last_peered":"2024-09-18T01:08:26.824247+0000","last_clean":"2024-09-18T01:08:26.824247+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:08:26.824247+0000","last_undegraded":"2024-09-18T01:08:26.824247+0000","last_fullsized":"2024-09-18T01:08:26.824247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:28.156878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000445952,"stat_sum":{"num_bytes":36137027,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":740,"num_read_kb":7834,"num_write":236,"num_write_kb":6097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.12","version":"77'198","reported_seq":1791,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783585+0000","last_change":"2024-09-18T01:08:24.551727+0000","last_active":"2024-09-18T01:08:26.783585+0000","last_peered":"2024-09-18T01:08:26.783585+0000","last_clean":"2024-09-18T01:08:26.783585+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:08:26.783585+0000","last_undegraded":"2024-09-18T01:08:26.783585+0000","last_fullsized":"2024-09-18T01:08:26.783585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:54:12.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076632200000000005,"stat_sum":{"num_bytes":13862708,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1956,"num_read_kb":16811,"num_write":238,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.13","version":"79'131","reported_seq":728,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788929+0000","last_change":"2024-09-18T01:08:24.557043+0000","last_active":"2024-09-18T01:08:26.788929+0000","last_peered":"2024-09-18T01:08:26.788929+0000","last_clean":"2024-09-18T01:08:26.788929+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:08:26.788929+0000","last_undegraded":"2024-09-18T01:08:26.788929+0000","last_fullsized":"2024-09-18T01:08:26.788929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:19.524756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074465000000000002,"stat_sum":{"num_bytes":5045608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":863,"num_read_kb":4343,"num_write":115,"num_write_kb":3213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":677530155,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":46770,"num_read_kb":645787,"num_write":9122,"num_write_kb":404260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":467816448,"data_stored":466136698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5749,"ondisk_log_size":5749,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102372,"kb_used_data":67020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94269468,"statfs":{"total":96636764160,"available":96531935232,"internally_reserved":0,"allocated":68628480,"data_stored":67953124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50244,"kb_used_data":19056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321596,"statfs":{"total":96636764160,"available":96585314304,"internally_reserved":0,"allocated":19513344,"data_stored":18953870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113452,"kb_used_data":78104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94258388,"statfs":{"total":96636764160,"available":96520589312,"internally_reserved":0,"allocated":79978496,"data_stored":79233894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60268,"kb_used_data":29072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311572,"statfs":{"total":96636764160,"available":96575049728,"internally_reserved":0,"allocated":29769728,"data_stored":29151169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74168,"kb_used_data":42980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297672,"statfs":{"total":96636764160,"available":96560816128,"internally_reserved":0,"allocated":44011520,"data_stored":43292766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100140,"kb_used_data":64792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271700,"statfs":{"total":96636764160,"available":96534220800,"internally_reserved":0,"allocated":66347008,"data_stored":65604044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125172,"kb_used_data":89828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94246668,"statfs":{"total":96636764160,"available":96508588032,"internally_reserved":0,"allocated":91983872,"data_stored":91195749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109664,"kb_used_data":74308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94262176,"statfs":{"total":96636764160,"available":96524468224,"internally_reserved":0,"allocated":76091392,"data_stored":75341649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64831488,"data_stored":64583681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43085824,"data_stored":42862752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28848128,"data_stored":28721174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":75206656,"data_stored":74930399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":91086848,"data_stored":90778176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":79052800,"data_stored":78803880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":18591744,"data_stored":18523875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":67112960,"data_stored":66932761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:08:28.180 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:08:28.182 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:28.204 INFO:teuthology.orchestra.run.smithi120.stdout:1532 write 0xc87744b thru 0xc881fee (0xaba4 bytes) 2024-09-18T01:08:28.217 INFO:teuthology.orchestra.run.smithi120.stdout:1533 clone 18 order 19 su 131072 sc 5 2024-09-18T01:08:28.227 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:28.226+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:28.285 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.285 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.293 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:28.296 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:28.294+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:28.397 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:28.396+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:28.437 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.438 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.447 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:28.482 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:08:28.494 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-18T01:08:27.778696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":678120542,"num_objects":696,"num_object_clones":0,"num_object_copies":1392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":696,"num_whiteouts":0,"num_read":46887,"num_read_kb":645888,"num_write":9283,"num_write_kb":406678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5869,"ondisk_log_size":5869,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":735480,"kb_used_data":465160,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754239240,"statfs":{"total":773094113280,"available":772340981760,"internally_reserved":0,"allocated":476323840,"data_stored":470726265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70848002,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1992,"num_read_kb":6477,"num_write":382,"num_write_kb":5216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021912"},"pg_stats":[{"pgid":"3.1f","version":"77'191","reported_seq":1577,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783725+0000","last_change":"2024-09-18T01:08:24.552111+0000","last_active":"2024-09-18T01:08:26.783725+0000","last_peered":"2024-09-18T01:08:26.783725+0000","last_clean":"2024-09-18T01:08:26.783725+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:08:26.783725+0000","last_undegraded":"2024-09-18T01:08:26.783725+0000","last_fullsized":"2024-09-18T01:08:26.783725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:53:24.429489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010483669999999999,"stat_sum":{"num_bytes":20862296,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1758,"num_read_kb":19496,"num_write":226,"num_write_kb":17312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1e","version":"74'90","reported_seq":724,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.546374+0000","last_change":"2024-09-18T01:08:24.546569+0000","last_active":"2024-09-18T01:08:24.546374+0000","last_peered":"2024-09-18T01:08:24.546374+0000","last_clean":"2024-09-18T01:08:24.546374+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:08:24.546374+0000","last_undegraded":"2024-09-18T01:08:24.546374+0000","last_fullsized":"2024-09-18T01:08:24.546374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:27.427417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285209999999999,"stat_sum":{"num_bytes":10656621,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":508,"num_read_kb":16488,"num_write":175,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1d","version":"79'120","reported_seq":1228,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.803490+0000","last_change":"2024-09-18T01:08:24.697488+0000","last_active":"2024-09-18T01:08:26.803490+0000","last_peered":"2024-09-18T01:08:26.803490+0000","last_clean":"2024-09-18T01:08:26.803490+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:08:26.803490+0000","last_undegraded":"2024-09-18T01:08:26.803490+0000","last_fullsized":"2024-09-18T01:08:26.803490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:18.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048755899999999999,"stat_sum":{"num_bytes":30508568,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":870,"num_read_kb":26307,"num_write":211,"num_write_kb":28930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1c","version":"78'105","reported_seq":833,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.841188+0000","last_change":"2024-09-18T01:08:24.697177+0000","last_active":"2024-09-18T01:08:26.841188+0000","last_peered":"2024-09-18T01:08:26.841188+0000","last_clean":"2024-09-18T01:08:26.841188+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:08:26.841188+0000","last_undegraded":"2024-09-18T01:08:26.841188+0000","last_fullsized":"2024-09-18T01:08:26.841188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:10.636440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046942400000000002,"stat_sum":{"num_bytes":21004077,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":530,"num_read_kb":21423,"num_write":130,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1b","version":"79'429","reported_seq":2086,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.831977+0000","last_change":"2024-09-18T01:08:24.552678+0000","last_active":"2024-09-18T01:08:26.831977+0000","last_peered":"2024-09-18T01:08:26.831977+0000","last_clean":"2024-09-18T01:08:26.831977+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:08:26.831977+0000","last_undegraded":"2024-09-18T01:08:26.831977+0000","last_fullsized":"2024-09-18T01:08:26.831977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:44.646917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012655590000000001,"stat_sum":{"num_bytes":24953907,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3261,"num_read_kb":26179,"num_write":704,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.1a","version":"74'109","reported_seq":1364,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:25.616448+0000","last_change":"2024-09-18T01:08:24.548295+0000","last_active":"2024-09-18T01:08:25.616448+0000","last_peered":"2024-09-18T01:08:25.616448+0000","last_clean":"2024-09-18T01:08:25.616448+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:08:25.616448+0000","last_undegraded":"2024-09-18T01:08:25.616448+0000","last_fullsized":"2024-09-18T01:08:25.616448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:26.875969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000409963,"stat_sum":{"num_bytes":15596178,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1335,"num_read_kb":20763,"num_write":173,"num_write_kb":11453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.19","version":"77'102","reported_seq":1269,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.824786+0000","last_change":"2024-09-18T01:08:24.547399+0000","last_active":"2024-09-18T01:08:26.824786+0000","last_peered":"2024-09-18T01:08:26.824786+0000","last_clean":"2024-09-18T01:08:26.824786+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:08:26.824786+0000","last_undegraded":"2024-09-18T01:08:26.824786+0000","last_fullsized":"2024-09-18T01:08:26.824786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:19:43.484695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052836400000000003,"stat_sum":{"num_bytes":28760688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":954,"num_read_kb":17758,"num_write":137,"num_write_kb":10788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.18","version":"73'124","reported_seq":2246,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545769+0000","last_change":"2024-09-18T01:08:24.545769+0000","last_active":"2024-09-18T01:08:24.545769+0000","last_peered":"2024-09-18T01:08:24.545769+0000","last_clean":"2024-09-18T01:08:24.545769+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:08:24.545769+0000","last_undegraded":"2024-09-18T01:08:24.545769+0000","last_fullsized":"2024-09-18T01:08:24.545769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:29.187547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091264099999999995,"stat_sum":{"num_bytes":29763459,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1964,"num_read_kb":29968,"num_write":175,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.17","version":"73'131","reported_seq":1065,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783784+0000","last_change":"2024-09-18T01:08:24.552789+0000","last_active":"2024-09-18T01:08:26.783784+0000","last_peered":"2024-09-18T01:08:26.783784+0000","last_clean":"2024-09-18T01:08:26.783784+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:08:26.783784+0000","last_undegraded":"2024-09-18T01:08:26.783784+0000","last_fullsized":"2024-09-18T01:08:26.783784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:43.664728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011101629999999999,"stat_sum":{"num_bytes":29119086,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":893,"num_read_kb":20345,"num_write":228,"num_write_kb":11682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.16","version":"71'86","reported_seq":1054,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788711+0000","last_change":"2024-09-18T01:08:24.697170+0000","last_active":"2024-09-18T01:08:26.788711+0000","last_peered":"2024-09-18T01:08:26.788711+0000","last_clean":"2024-09-18T01:08:26.788711+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:08:26.788711+0000","last_undegraded":"2024-09-18T01:08:26.788711+0000","last_fullsized":"2024-09-18T01:08:26.788711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:14.837855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050167300000000001,"stat_sum":{"num_bytes":25668979,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":762,"num_read_kb":14832,"num_write":124,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.15","version":"79'146","reported_seq":2232,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788670+0000","last_change":"2024-09-18T01:08:24.557082+0000","last_active":"2024-09-18T01:08:26.788670+0000","last_peered":"2024-09-18T01:08:26.788670+0000","last_clean":"2024-09-18T01:08:26.788670+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:08:26.788670+0000","last_undegraded":"2024-09-18T01:08:26.788670+0000","last_fullsized":"2024-09-18T01:08:26.788670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:12.588382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057264499999999999,"stat_sum":{"num_bytes":30703508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2078,"num_read_kb":39372,"num_write":170,"num_write_kb":17302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.14","version":"74'325","reported_seq":881,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.547759+0000","last_change":"2024-09-18T01:08:24.547945+0000","last_active":"2024-09-18T01:08:24.547759+0000","last_peered":"2024-09-18T01:08:24.547759+0000","last_clean":"2024-09-18T01:08:24.547759+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:08:24.547759+0000","last_undegraded":"2024-09-18T01:08:24.547759+0000","last_fullsized":"2024-09-18T01:08:24.547759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:38.320977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082267699999999996,"stat_sum":{"num_bytes":6357140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1616,"num_read_kb":11155,"num_write":529,"num_write_kb":5563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788767+0000","last_change":"2024-09-18T01:07:58.635173+0000","last_active":"2024-09-18T01:08:26.788767+0000","last_peered":"2024-09-18T01:08:26.788767+0000","last_clean":"2024-09-18T01:08:26.788767+0000","last_became_active":"2024-09-18T01:07:58.634732+0000","last_became_peered":"2024-09-18T01:07:58.634732+0000","last_unstale":"2024-09-18T01:08:26.788767+0000","last_undegraded":"2024-09-18T01:08:26.788767+0000","last_fullsized":"2024-09-18T01:08:26.788767+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:52:17.327995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"74'139","reported_seq":1410,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783559+0000","last_change":"2024-09-18T01:08:24.552000+0000","last_active":"2024-09-18T01:08:26.783559+0000","last_peered":"2024-09-18T01:08:26.783559+0000","last_clean":"2024-09-18T01:08:26.783559+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:08:26.783559+0000","last_undegraded":"2024-09-18T01:08:26.783559+0000","last_fullsized":"2024-09-18T01:08:26.783559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:39:08.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093537100000000005,"stat_sum":{"num_bytes":10101919,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1853,"num_read_kb":14920,"num_write":236,"num_write_kb":9399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.778146+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:08:26.778146+0000","last_peered":"2024-09-18T01:08:26.778146+0000","last_clean":"2024-09-18T01:08:26.778146+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:08:26.778146+0000","last_undegraded":"2024-09-18T01:08:26.778146+0000","last_fullsized":"2024-09-18T01:08:26.778146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"64'185","reported_seq":846,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545857+0000","last_change":"2024-09-18T01:08:24.546273+0000","last_active":"2024-09-18T01:08:24.545857+0000","last_peered":"2024-09-18T01:08:24.545857+0000","last_clean":"2024-09-18T01:08:24.545857+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:08:24.545857+0000","last_undegraded":"2024-09-18T01:08:24.545857+0000","last_fullsized":"2024-09-18T01:08:24.545857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:03.212528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013889950000000001,"stat_sum":{"num_bytes":9748480,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1031,"num_read_kb":17129,"num_write":349,"num_write_kb":12969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788913+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:08:26.788913+0000","last_peered":"2024-09-18T01:08:26.788913+0000","last_clean":"2024-09-18T01:08:26.788913+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:08:26.788913+0000","last_undegraded":"2024-09-18T01:08:26.788913+0000","last_fullsized":"2024-09-18T01:08:26.788913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'134","reported_seq":1859,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.821066+0000","last_change":"2024-09-18T01:08:24.547682+0000","last_active":"2024-09-18T01:08:26.821066+0000","last_peered":"2024-09-18T01:08:26.821066+0000","last_clean":"2024-09-18T01:08:26.821066+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:08:26.821066+0000","last_undegraded":"2024-09-18T01:08:26.821066+0000","last_fullsized":"2024-09-18T01:08:26.821066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:10.763832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000415905,"stat_sum":{"num_bytes":30193482,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1829,"num_read_kb":28850,"num_write":143,"num_write_kb":9207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.778046+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:08:26.778046+0000","last_peered":"2024-09-18T01:08:26.778046+0000","last_clean":"2024-09-18T01:08:26.778046+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:08:26.778046+0000","last_undegraded":"2024-09-18T01:08:26.778046+0000","last_fullsized":"2024-09-18T01:08:26.778046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"72'256","reported_seq":2324,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788865+0000","last_change":"2024-09-18T01:08:24.697590+0000","last_active":"2024-09-18T01:08:26.788865+0000","last_peered":"2024-09-18T01:08:26.788865+0000","last_clean":"2024-09-18T01:08:26.788865+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:08:26.788865+0000","last_undegraded":"2024-09-18T01:08:26.788865+0000","last_fullsized":"2024-09-18T01:08:26.788865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:48.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056296500000000001,"stat_sum":{"num_bytes":41103835,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2514,"num_read_kb":38721,"num_write":498,"num_write_kb":23286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.3","version":"74'210","reported_seq":1276,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.547823+0000","last_change":"2024-09-18T01:08:24.548053+0000","last_active":"2024-09-18T01:08:24.547823+0000","last_peered":"2024-09-18T01:08:24.547823+0000","last_clean":"2024-09-18T01:08:24.547823+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:08:24.547823+0000","last_undegraded":"2024-09-18T01:08:24.547823+0000","last_fullsized":"2024-09-18T01:08:24.547823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:37.905731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052537600000000003,"stat_sum":{"num_bytes":9438348,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1619,"num_read_kb":15315,"num_write":294,"num_write_kb":7339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":136,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788458+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:08:26.788458+0000","last_peered":"2024-09-18T01:08:26.788458+0000","last_clean":"2024-09-18T01:08:26.788458+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:08:26.788458+0000","last_undegraded":"2024-09-18T01:08:26.788458+0000","last_fullsized":"2024-09-18T01:08:26.788458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"74'136","reported_seq":1799,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.777615+0000","last_change":"2024-09-18T01:08:24.548079+0000","last_active":"2024-09-18T01:08:26.777615+0000","last_peered":"2024-09-18T01:08:26.777615+0000","last_clean":"2024-09-18T01:08:26.777615+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:08:26.777615+0000","last_undegraded":"2024-09-18T01:08:26.777615+0000","last_fullsized":"2024-09-18T01:08:26.777615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:39.957103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078098800000000004,"stat_sum":{"num_bytes":28013935,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1686,"num_read_kb":36481,"num_write":297,"num_write_kb":17730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.809288+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:08:26.809288+0000","last_peered":"2024-09-18T01:08:26.809288+0000","last_clean":"2024-09-18T01:08:26.809288+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:08:26.809288+0000","last_undegraded":"2024-09-18T01:08:26.809288+0000","last_fullsized":"2024-09-18T01:08:26.809288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"77'156","reported_seq":1397,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.825939+0000","last_change":"2024-09-18T01:08:24.551863+0000","last_active":"2024-09-18T01:08:26.825939+0000","last_peered":"2024-09-18T01:08:26.825939+0000","last_clean":"2024-09-18T01:08:26.825939+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:08:26.825939+0000","last_undegraded":"2024-09-18T01:08:26.825939+0000","last_fullsized":"2024-09-18T01:08:26.825939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:48.860383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090600000000004,"stat_sum":{"num_bytes":32397194,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1280,"num_read_kb":13078,"num_write":158,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788305+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:08:26.788305+0000","last_peered":"2024-09-18T01:08:26.788305+0000","last_clean":"2024-09-18T01:08:26.788305+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:08:26.788305+0000","last_undegraded":"2024-09-18T01:08:26.788305+0000","last_fullsized":"2024-09-18T01:08:26.788305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"71'150","reported_seq":1307,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.694637+0000","last_change":"2024-09-18T01:08:24.546252+0000","last_active":"2024-09-18T01:08:24.694637+0000","last_peered":"2024-09-18T01:08:24.694637+0000","last_clean":"2024-09-18T01:08:24.694637+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:08:24.694637+0000","last_undegraded":"2024-09-18T01:08:24.694637+0000","last_fullsized":"2024-09-18T01:08:24.694637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:35:38.245719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011962030000000001,"stat_sum":{"num_bytes":19203626,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1272,"num_read_kb":12774,"num_write":183,"num_write_kb":17754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788499+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:08:26.788499+0000","last_peered":"2024-09-18T01:08:26.788499+0000","last_clean":"2024-09-18T01:08:26.788499+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:08:26.788499+0000","last_undegraded":"2024-09-18T01:08:26.788499+0000","last_fullsized":"2024-09-18T01:08:26.788499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":286,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788379+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:08:26.788379+0000","last_peered":"2024-09-18T01:08:26.788379+0000","last_clean":"2024-09-18T01:08:26.788379+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:08:26.788379+0000","last_undegraded":"2024-09-18T01:08:26.788379+0000","last_fullsized":"2024-09-18T01:08:26.788379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'480","reported_seq":1447,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.546523+0000","last_change":"2024-09-18T01:08:24.546523+0000","last_active":"2024-09-18T01:08:24.546523+0000","last_peered":"2024-09-18T01:08:24.546523+0000","last_clean":"2024-09-18T01:08:24.546523+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:08:24.546523+0000","last_undegraded":"2024-09-18T01:08:24.546523+0000","last_fullsized":"2024-09-18T01:08:24.546523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:04:51.061212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010885619999999999,"stat_sum":{"num_bytes":8273963,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2742,"num_read_kb":12424,"num_write":926,"num_write_kb":5521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.9","version":"77'73","reported_seq":699,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:25.584672+0000","last_change":"2024-09-18T01:08:24.547523+0000","last_active":"2024-09-18T01:08:25.584672+0000","last_peered":"2024-09-18T01:08:25.584672+0000","last_clean":"2024-09-18T01:08:25.584672+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:08:25.584672+0000","last_undegraded":"2024-09-18T01:08:25.584672+0000","last_fullsized":"2024-09-18T01:08:25.584672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:50.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046679600000000001,"stat_sum":{"num_bytes":19645466,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":396,"num_read_kb":19056,"num_write":79,"num_write_kb":8783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.a","version":"71'101","reported_seq":1280,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.548015+0000","last_change":"2024-09-18T01:08:24.548165+0000","last_active":"2024-09-18T01:08:24.548015+0000","last_peered":"2024-09-18T01:08:24.548015+0000","last_clean":"2024-09-18T01:08:24.548015+0000","last_became_active":"2024-09-18T01:07:58.640172+0000","last_became_peered":"2024-09-18T01:07:58.640172+0000","last_unstale":"2024-09-18T01:08:24.548015+0000","last_undegraded":"2024-09-18T01:08:24.548015+0000","last_fullsized":"2024-09-18T01:08:24.548015+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:33.051870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060734799999999996,"stat_sum":{"num_bytes":42510450,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":948,"num_read_kb":16793,"num_write":161,"num_write_kb":7285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.b","version":"74'149","reported_seq":1079,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-18T01:08:24.545428+0000","last_change":"2024-09-18T01:08:24.545761+0000","last_active":"2024-09-18T01:08:24.545428+0000","last_peered":"2024-09-18T01:08:24.545428+0000","last_clean":"2024-09-18T01:08:24.545428+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:08:24.545428+0000","last_undegraded":"2024-09-18T01:08:24.545428+0000","last_fullsized":"2024-09-18T01:08:24.545428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:43.756298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089125599999999999,"stat_sum":{"num_bytes":12508096,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":978,"num_read_kb":18864,"num_write":243,"num_write_kb":8595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.c","version":"79'479","reported_seq":1882,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788595+0000","last_change":"2024-09-18T01:08:24.697483+0000","last_active":"2024-09-18T01:08:26.788595+0000","last_peered":"2024-09-18T01:08:26.788595+0000","last_clean":"2024-09-18T01:08:26.788595+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:08:26.788595+0000","last_undegraded":"2024-09-18T01:08:26.788595+0000","last_fullsized":"2024-09-18T01:08:26.788595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:22:21.343183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000613183,"stat_sum":{"num_bytes":13484728,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":3089,"num_read_kb":17421,"num_write":937,"num_write_kb":7766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.d","version":"71'111","reported_seq":1195,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788792+0000","last_change":"2024-09-18T01:08:24.557303+0000","last_active":"2024-09-18T01:08:26.788792+0000","last_peered":"2024-09-18T01:08:26.788792+0000","last_clean":"2024-09-18T01:08:26.788792+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:08:26.788792+0000","last_undegraded":"2024-09-18T01:08:26.788792+0000","last_fullsized":"2024-09-18T01:08:26.788792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:27:52.623740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082317600000000001,"stat_sum":{"num_bytes":17517309,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1092,"num_read_kb":22631,"num_write":139,"num_write_kb":15375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.e","version":"79'335","reported_seq":2297,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.789354+0000","last_change":"2024-09-18T01:08:24.558122+0000","last_active":"2024-09-18T01:08:26.789354+0000","last_peered":"2024-09-18T01:08:26.789354+0000","last_clean":"2024-09-18T01:08:26.789354+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:08:26.789354+0000","last_undegraded":"2024-09-18T01:08:26.789354+0000","last_fullsized":"2024-09-18T01:08:26.789354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:36.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010028979999999999,"stat_sum":{"num_bytes":33326407,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2722,"num_read_kb":41024,"num_write":554,"num_write_kb":26929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.f","version":"74'135","reported_seq":1062,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788694+0000","last_change":"2024-09-18T01:08:24.557394+0000","last_active":"2024-09-18T01:08:26.788694+0000","last_peered":"2024-09-18T01:08:26.788694+0000","last_clean":"2024-09-18T01:08:26.788694+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:08:26.788694+0000","last_undegraded":"2024-09-18T01:08:26.788694+0000","last_fullsized":"2024-09-18T01:08:26.788694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:45.831846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050360900000000002,"stat_sum":{"num_bytes":9284284,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1007,"num_read_kb":13572,"num_write":220,"num_write_kb":6993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.10","version":"74'110","reported_seq":882,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783635+0000","last_change":"2024-09-18T01:08:24.552239+0000","last_active":"2024-09-18T01:08:26.783635+0000","last_peered":"2024-09-18T01:08:26.783635+0000","last_clean":"2024-09-18T01:08:26.783635+0000","last_became_active":"2024-09-18T01:07:58.634581+0000","last_became_peered":"2024-09-18T01:07:58.634581+0000","last_unstale":"2024-09-18T01:08:26.783635+0000","last_undegraded":"2024-09-18T01:08:26.783635+0000","last_fullsized":"2024-09-18T01:08:26.783635+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:26.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083914099999999995,"stat_sum":{"num_bytes":11778783,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":624,"num_read_kb":13660,"num_write":134,"num_write_kb":10782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.11","version":"77'123","reported_seq":909,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.824247+0000","last_change":"2024-09-18T01:08:24.556742+0000","last_active":"2024-09-18T01:08:26.824247+0000","last_peered":"2024-09-18T01:08:26.824247+0000","last_clean":"2024-09-18T01:08:26.824247+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:08:26.824247+0000","last_undegraded":"2024-09-18T01:08:26.824247+0000","last_fullsized":"2024-09-18T01:08:26.824247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:28.156878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000445952,"stat_sum":{"num_bytes":36137027,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":740,"num_read_kb":7834,"num_write":236,"num_write_kb":6097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.12","version":"77'198","reported_seq":1791,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.783585+0000","last_change":"2024-09-18T01:08:24.551727+0000","last_active":"2024-09-18T01:08:26.783585+0000","last_peered":"2024-09-18T01:08:26.783585+0000","last_clean":"2024-09-18T01:08:26.783585+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:08:26.783585+0000","last_undegraded":"2024-09-18T01:08:26.783585+0000","last_fullsized":"2024-09-18T01:08:26.783585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:54:12.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076632200000000005,"stat_sum":{"num_bytes":13862708,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1956,"num_read_kb":16811,"num_write":238,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]},{"pgid":"3.13","version":"79'131","reported_seq":728,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T01:08:26.788929+0000","last_change":"2024-09-18T01:08:24.557043+0000","last_active":"2024-09-18T01:08:26.788929+0000","last_peered":"2024-09-18T01:08:26.788929+0000","last_clean":"2024-09-18T01:08:26.788929+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:08:26.788929+0000","last_undegraded":"2024-09-18T01:08:26.788929+0000","last_fullsized":"2024-09-18T01:08:26.788929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:19.524756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074465000000000002,"stat_sum":{"num_bytes":5045608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":863,"num_read_kb":4343,"num_write":115,"num_write_kb":3213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":677530155,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":46770,"num_read_kb":645787,"num_write":9122,"num_write_kb":404260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":467816448,"data_stored":466136698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5749,"ondisk_log_size":5749,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102372,"kb_used_data":67020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94269468,"statfs":{"total":96636764160,"available":96531935232,"internally_reserved":0,"allocated":68628480,"data_stored":67953124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50244,"kb_used_data":19056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321596,"statfs":{"total":96636764160,"available":96585314304,"internally_reserved":0,"allocated":19513344,"data_stored":18953870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113452,"kb_used_data":78104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94258388,"statfs":{"total":96636764160,"available":96520589312,"internally_reserved":0,"allocated":79978496,"data_stored":79233894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60268,"kb_used_data":29072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311572,"statfs":{"total":96636764160,"available":96575049728,"internally_reserved":0,"allocated":29769728,"data_stored":29151169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74168,"kb_used_data":42980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297672,"statfs":{"total":96636764160,"available":96560816128,"internally_reserved":0,"allocated":44011520,"data_stored":43292766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100140,"kb_used_data":64792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271700,"statfs":{"total":96636764160,"available":96534220800,"internally_reserved":0,"allocated":66347008,"data_stored":65604044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125172,"kb_used_data":89828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94246668,"statfs":{"total":96636764160,"available":96508588032,"internally_reserved":0,"allocated":91983872,"data_stored":91195749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109664,"kb_used_data":74308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94262176,"statfs":{"total":96636764160,"available":96524468224,"internally_reserved":0,"allocated":76091392,"data_stored":75341649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64831488,"data_stored":64583681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43085824,"data_stored":42862752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28848128,"data_stored":28721174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":75206656,"data_stored":74930399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":91086848,"data_stored":90778176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":79052800,"data_stored":78803880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":18591744,"data_stored":18523875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":67112960,"data_stored":66932761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:08:28.496 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:08:28.496 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:28.496 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T01:08:28.496 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-18T01:08:28.497 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T01:08:28.530 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:28.529+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:28.589 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.590 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.599 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:28.599 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:28.596+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:28.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:28.700 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:28.699+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:28.744 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.744 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:28.795 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:28.796 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:28.800 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:28.798+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:28.818 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:28.818 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:28.835 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:28.836 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:28.852 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:28.852 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:28.870 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:28.870 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:28.886 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:28.887 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:28.901 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:28.901 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:28.902 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:28.899+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:28.902 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:28.903 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:28.910 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:28.919 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:28.919 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:29.002 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:29.001+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:29.013 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:29.013 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:29.013 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:29.013 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:29.013 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:29.022 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:29.103 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:29.102+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.266988249999997, 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.873808951999997, 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.507077652, 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.227690502999998, 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.535368779000002, 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.535353682, 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.535342532000001, 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.535332895000003, 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.531420847, 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.467919597000002, 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43586 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5816320~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769936878999999, 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015218535, 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43586 2024-09-18T01:08:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2499999999999999e-07 2024-09-18T01:08:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394861+0000", 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394864+0000", 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:08:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394867+0000", 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.409942+0000", 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015075312 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410015+0000", 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3090000000000007e-05 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410079+0000", 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3126999999999994e-05 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43587 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5980160~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769927013, 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015359998, 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43587 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394871+0000", 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4200000000000004e-07 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394874+0000", 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394876+0000", 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4399999999999999e-06 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410105+0000", 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015228759 2024-09-18T01:08:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410155+0000", 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9987000000000001e-05 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410230+0000", 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4697999999999995e-05 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43588 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6144000~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769917649, 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015489108999999999, 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43588 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394880+0000", 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394884+0000", 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6829999999999999e-06 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394888+0000", 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3409999999999999e-06 2024-09-18T01:08:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394890+0000", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.543e-06 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410258+0000", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015368371 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410303+0000", 2024-09-18T01:08:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410368+0000", 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5747999999999995e-05 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43589 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6307840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769903080999999, 2024-09-18T01:08:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015665806000000001, 2024-09-18T01:08:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43589 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394895+0000", 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0079999999999999e-06 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394904+0000", 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.234e-06 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394909+0000", 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410406+0000", 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015497273000000001 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410486+0000", 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9722999999999995e-05 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410560+0000", 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4008e-05 2024-09-18T01:08:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43590 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6471680~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769883999000001, 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01587189, 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43590 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8500000000000002e-07 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394915+0000", 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3239999999999999e-06 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394918+0000", 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8789999999999996e-06 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394921+0000", 2024-09-18T01:08:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410601+0000", 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015679648000000001 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410683+0000", 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2826999999999998e-05 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410785+0000", 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101365 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43591 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6635520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769872775, 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015982698, 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43591 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394925+0000", 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.006e-06 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394928+0000", 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394931+0000", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5720000000000001e-06 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410803+0000", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015871962999999999 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410847+0000", 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3912000000000002e-05 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410907+0000", 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0025999999999998e-05 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43592 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6799360~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.769848417999999, 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016170249000000001, 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43592 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394950+0000", 2024-09-18T01:08:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.161e-06 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394955+0000", 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9089999999999999e-06 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394958+0000", 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.593e-06 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410941+0000", 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015983431999999999 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411019+0000", 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8325999999999995e-05 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411119+0000", 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9463999999999995e-05 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50421 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9318563470000001, 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020565198, 2024-09-18T01:08:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50421 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232946+0000", 2024-09-18T01:08:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3209999999999998e-06 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232956+0000", 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0368e-05 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232963+0000", 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2400000000000001e-06 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233001+0000", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7298000000000001e-05 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233052+0000", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1659000000000001e-05 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.236268+0000", 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003215894 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253414+0000", 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017146122999999999 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253429+0000", 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4674e-05 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253468+0000", 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.896e-05 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253506+0000", 2024-09-18T01:08:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7793999999999998e-05 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50514 3.11 3:8aff915a:::rbd_data.11761711706f.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7690058749999999, 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015404816999999999, 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50514 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395792+0000", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6300000000000005e-07 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395796+0000", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4490000000000001e-06 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395805+0000", 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8580000000000006e-06 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395810+0000", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395844+0000", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769000000000001e-05 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395895+0000", 2024-09-18T01:08:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0930000000000002e-05 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.398279+0000", 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023842569999999999 2024-09-18T01:08:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411026+0000", 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012747 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411078+0000", 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2296000000000002e-05 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411143+0000", 2024-09-18T01:08:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4561999999999996e-05 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411196+0000", 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3152999999999997e-05 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50582 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.6410285789999999, 2024-09-18T01:08:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027666276, 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50582 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523769+0000", 2024-09-18T01:08:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8599999999999997e-07 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523774+0000", 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523783+0000", 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5149999999999995e-06 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523788+0000", 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523838+0000", 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9438999999999999e-05 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523930+0000", 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1978000000000006e-05 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.528122+0000", 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004192749 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551279+0000", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023157015999999999 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551339+0000", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9410000000000002e-05 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551380+0000", 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0927000000000003e-05 2024-09-18T01:08:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551435+0000", 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4846999999999999e-05 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:29.292 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:29.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:29.294 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:29.292+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:29.394 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:29.393+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:29.398 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:29.398 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:29.398 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:29.398 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:29.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:29.496 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:29.494+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:29.509 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:29.510 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:29.518 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T01:08:29.604 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:29.601+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:29.674 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:29.674 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:29.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:29.674 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.085804109999998, 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:29.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:29.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:29.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.082170357000003, 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:29.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.082146788999999, 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:29.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:29.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.080571386999999, 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:29.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:29.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:29.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.076138296000003, 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.697 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:29.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:29.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.074816118999998, 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.703 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:29.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:29.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.070762307999999, 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:29.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.045073879, 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.014535674000001, 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.014045981999999, 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.949686285999999, 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778648654000001, 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778619888000001, 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778596641, 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778576638000001, 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778557995, 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778539453, 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778518866999999, 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.778474596999999, 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.777494404999999, 2024-09-18T01:08:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:29.781 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:29.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:29.783 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:29.781+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:29.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:08:29.883 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:29.881+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:29.930 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:29.931 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:29.931 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:29.931 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:29.940 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:29.983 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:29.983+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:30.084 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.084 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:30.084 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:30.084 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:30.084 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.084 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:30.082+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:30.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:30.185 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:30.184+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:30.236 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.236 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056297246, 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.237 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:30.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:30.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056280678, 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056264491999997, 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:30.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.251 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:30.252 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056252923000002, 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.253 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:30.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:30.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:30.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:30.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.258 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056243557000002, 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:30.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056231373999999, 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.264 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:30.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056215678999997, 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:30.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:30.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056198524999999, 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.274 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:30.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056176358999998, 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056159909000002, 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056150455000001, 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056116404000001, 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056096680000003, 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056078442999997, 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056057868000003, 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056038641000001, 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056019288000002, 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.056000632999996, 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.047218684000001, 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:50519 3.19 3:9a8c2274:::rbd_data.11761711706f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8157675749999997, 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026218866, 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 50519 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418749+0000", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.85e-07 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418754+0000", 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2910000000000001e-06 2024-09-18T01:08:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418764+0000", 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418769+0000", 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1789999999999999e-06 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418814+0000", 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4350000000000001e-05 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418863+0000", 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9264000000000001e-05 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.423079+0000", 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042156269999999996 2024-09-18T01:08:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444883+0000", 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021804975000000001 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444896+0000", 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2344000000000001e-05 2024-09-18T01:08:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444929+0000", 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.269e-05 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444967+0000", 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8818e-05 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.344 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.345 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:30.346 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:30.344+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:30.446 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:30.445+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:30.448 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.448 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:30.448 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:30.448 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.457 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:30.547 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:30.546+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:30.548 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:08:30.567 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.567 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:30.567 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:30.567 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:30.567 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.578 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:30.649 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:30.647+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:30.722 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:30.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.781058290000004, 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:30.727 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.653130249999997, 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:30.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:30.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:30.732 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.602756190000001, 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:30.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.512862974000001, 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:30.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:30.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.511267226000001, 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:30.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:30.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:30.746 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.510456607000002, 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:30.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.127083796000001, 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:30.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:30.755 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.563483738999999, 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.545722652999999, 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:30.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.543065949999999, 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:30.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.960418406999999, 2024-09-18T01:08:30.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:30.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.546104948, 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:30.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.044405926, 2024-09-18T01:08:30.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:30.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:30.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.042829039000001, 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:30.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:30.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.013081449000001, 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:30.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.967438288, 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.790 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:30.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:30.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.857701963, 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:30.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.854048533, 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:49241 3.9 e73/23 3:95d710ee:::rbd_data.117667234fca.0000000000000001:head v 73'60, mlcod=64'58)", 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.142928467999999, 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034975370000000002, 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579263+0000", 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014074720000000001 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579278+0000", 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5509999999999999e-05 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579288+0000", 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8490000000000001e-06 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579364+0000", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5507999999999996e-05 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579389+0000", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5196999999999999e-05 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.581309+0000", 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001919802 2024-09-18T01:08:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.581352+0000", 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3738e-05 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:52190 3.4 e82/23 3:2587a2b7:::rbd_data.11762afc950e.0000000000000022:head v 82'135, mlcod=77'133)", 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.1178323429999999, 2024-09-18T01:08:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038334850000000002, 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602975+0000", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3629999999999999e-05 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602984+0000", 2024-09-18T01:08:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0820000000000005e-06 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602992+0000", 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6920000000000002e-06 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603100+0000", 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107977 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603124+0000", 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4544000000000001e-05 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606735+0000", 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036103580000000001 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606784+0000", 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9697000000000002e-05 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:30.814 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.816 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:30.817 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:30.815+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:30.916 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:30.915+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:30.919 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:30.920 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:30.928 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:30.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:31.018 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:31.017+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:31.036 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:31.036 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:31.036 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:31.036 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:31.036 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:31.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:31.057 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:31.058 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:31.075 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:31.075 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:31.091 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:31.091 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:31.107 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:31.107 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:31.118 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:31.117+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:31.124 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:31.141 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:31.153 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.153 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.158 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:31.158 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:31.162 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:31.175 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:31.175 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:31.220 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:31.219+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:31.278 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.278 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:31.329 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:31.328+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:31.421 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:31.419+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:31.432 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.432 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.441 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:31.522 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:31.520+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:31.544 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.544 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.554 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:31.622 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:31.621+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:31.658 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.659 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.667 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:31.691 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-18T01:08:31.723 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:31.721+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:31.813 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.813 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:31.823 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:31.821+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:31.924 INFO:teuthology.orchestra.run.smithi120.stdout:1534 trunc from 0xd4066b6 to 0xe4d4868 2024-09-18T01:08:31.925 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:31.922+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:31.925 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:31.925 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:31.934 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:31.938 INFO:teuthology.orchestra.run.smithi120.stdout:1535 write 0xed30912 thru 0xed35c96 (0x5385 bytes) 2024-09-18T01:08:31.944 INFO:teuthology.orchestra.run.smithi120.stdout:1536 writesame 0xb362da7 thru 0xb36ce56 data_size 0x18(0xa0b0 bytes) 2024-09-18T01:08:31.948 INFO:teuthology.orchestra.run.smithi120.stdout:1537 read 0x8d5d71c thru 0x8d665ab (0x8e90 bytes) 2024-09-18T01:08:31.948 INFO:teuthology.orchestra.run.smithi120.stdout:1538 punch from 0x33c5cd0 to 0x33c8cf7, (0x3027 bytes) 2024-09-18T01:08:31.949 INFO:teuthology.orchestra.run.smithi120.stdout:1539 punch from 0x456d284 to 0x45767c0, (0x953c bytes) 2024-09-18T01:08:31.949 INFO:teuthology.orchestra.run.smithi120.stdout:1540 read 0x8b2bf80 thru 0x8b32abf (0x6b40 bytes) 2024-09-18T01:08:31.949 INFO:teuthology.orchestra.run.smithi120.stdout:1541 writesame 0xcfb3e16 thru 0xcfc042a data_size 0x1(0xc615 bytes) 2024-09-18T01:08:31.974 INFO:teuthology.orchestra.run.smithi120.stdout:1542 read 0xc343223 thru 0xc34d694 (0xa472 bytes) 2024-09-18T01:08:31.975 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:31.982 INFO:teuthology.orchestra.run.smithi120.stdout:1544 trunc from 0xed35c97 to 0x8b72aa5 2024-09-18T01:08:32.010 INFO:teuthology.orchestra.run.smithi120.stdout:1546 read 0x5d3fc65 thru 0x5d458a9 (0x5c45 bytes) 2024-09-18T01:08:32.010 INFO:teuthology.orchestra.run.smithi120.stdout:1547 read 0x11ae45a thru 0x11afacc (0x1673 bytes) 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:1548 writesame 0xa257fe3 thru 0xa26529d data_size 0x1(0xd2bb bytes) 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:1549 read 0x8da43d3 thru 0x8dac5ae (0x81dc bytes) 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:1553 read 0x40bc836 thru 0x40c888f (0xc05a bytes) 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:1555 read 0x99b7227 thru 0x99b86be (0x1498 bytes) 2024-09-18T01:08:32.018 INFO:teuthology.orchestra.run.smithi120.stdout:1556 write 0x851b8ac thru 0x851dd0b (0x2460 bytes) 2024-09-18T01:08:32.023 INFO:teuthology.orchestra.run.smithi120.stdout:1557 read 0x62ea9d8 thru 0x62f6117 (0xb740 bytes) 2024-09-18T01:08:32.024 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.025 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:32.023+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:32.035 INFO:teuthology.orchestra.run.smithi120.stdout:1559 write 0xd25c7f9 thru 0xd2647e7 (0x7fef bytes) 2024-09-18T01:08:32.047 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.047 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.054 INFO:teuthology.orchestra.run.smithi120.stdout:1561 trunc from 0xd2647e8 to 0x3088133 2024-09-18T01:08:32.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:32.073 INFO:teuthology.orchestra.run.smithi120.stdout:1562 read 0x16fbd98 thru 0x16fee08 (0x3071 bytes) 2024-09-18T01:08:32.087 INFO:teuthology.orchestra.run.smithi120.stdout:1563 writesame 0xa16afbc thru 0xa16b2fd data_size 0x1(0x342 bytes) 2024-09-18T01:08:32.088 INFO:teuthology.orchestra.run.smithi120.stdout:1564 write 0x427fda2 thru 0x428fae6 (0xfd45 bytes) 2024-09-18T01:08:32.107 INFO:teuthology.orchestra.run.smithi120.stdout:1565 trunc from 0xa16b2fe to 0xc1459f 2024-09-18T01:08:32.126 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:32.125+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:32.163 INFO:teuthology.orchestra.run.smithi120.stdout:1566 write 0xd325055 thru 0xd3251b0 (0x15c bytes) 2024-09-18T01:08:32.166 INFO:teuthology.orchestra.run.smithi120.stdout:1567 writesame 0xb544bf8 thru 0xb550f4c data_size 0x49(0xc355 bytes) 2024-09-18T01:08:32.169 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.170 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.170 INFO:teuthology.orchestra.run.smithi120.stdout:1568 punch from 0x5073820 to 0x5075b49, (0x2329 bytes) 2024-09-18T01:08:32.170 INFO:teuthology.orchestra.run.smithi120.stdout:1570 writesame 0xc204de5 thru 0xc213b7b data_size 0x1(0xed97 bytes) 2024-09-18T01:08:32.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:32.202 INFO:teuthology.orchestra.run.smithi120.stdout:1571 writesame 0x664b750 thru 0x664ddd5 data_size 0x1(0x2686 bytes) 2024-09-18T01:08:32.220 INFO:teuthology.orchestra.run.smithi120.stdout:1573 trunc from 0xd3251b1 to 0x3fce40f 2024-09-18T01:08:32.226 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:32.224+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:32.231 INFO:teuthology.orchestra.run.smithi120.stdout:1574 punch from 0x2c16cb2 to 0x2c19737, (0x2a85 bytes) 2024-09-18T01:08:32.233 INFO:teuthology.orchestra.run.smithi120.stdout:1575 trunc from 0x3fce40f to 0x6b06318 2024-09-18T01:08:32.243 INFO:teuthology.orchestra.run.smithi120.stdout:1576 write 0x5fae4b0 thru 0x5fb8cc8 (0xa819 bytes) 2024-09-18T01:08:32.249 INFO:teuthology.orchestra.run.smithi120.stdout:1577 punch from 0x1b89017 to 0x1b91ea1, (0x8e8a bytes) 2024-09-18T01:08:32.249 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.249 INFO:teuthology.orchestra.run.smithi120.stdout:1579 read 0x466f8b0 thru 0x46761d2 (0x6923 bytes) 2024-09-18T01:08:32.249 INFO:teuthology.orchestra.run.smithi120.stdout:1580 write 0x268371d thru 0x26909d8 (0xd2bc bytes) 2024-09-18T01:08:32.262 INFO:teuthology.orchestra.run.smithi120.stdout:1581 write 0x8d31a5e thru 0x8d33b7c (0x211f bytes) 2024-09-18T01:08:32.279 INFO:teuthology.orchestra.run.smithi120.stdout:1582 write 0xeb310cd thru 0xeb3df8f (0xcec3 bytes) 2024-09-18T01:08:32.283 INFO:teuthology.orchestra.run.smithi120.stdout:1583 write 0x8197d23 thru 0x81a35fc (0xb8da bytes) 2024-09-18T01:08:32.289 INFO:teuthology.orchestra.run.smithi120.stdout:1584 punch from 0x9ae0008 to 0x9ae133f, (0x1337 bytes) 2024-09-18T01:08:32.290 INFO:teuthology.orchestra.run.smithi120.stdout:1585 write 0xb1f1026 thru 0xb1fe3eb (0xd3c6 bytes) 2024-09-18T01:08:32.297 INFO:teuthology.orchestra.run.smithi120.stdout:1586 punch from 0xcfc231f to 0xcfd12c0, (0xefa1 bytes) 2024-09-18T01:08:32.297 INFO:teuthology.orchestra.run.smithi120.stdout:1587 writesame 0xb13bc0 thru 0xb20ac8 data_size 0x27(0xcf09 bytes) 2024-09-18T01:08:32.298 INFO:teuthology.orchestra.run.smithi120.stdout:1589 writesame 0x78555ab thru 0x785d66e data_size 0x29(0x80c4 bytes) 2024-09-18T01:08:32.305 INFO:teuthology.orchestra.run.smithi120.stdout:1590 trunc from 0xeb3df90 to 0xc47294f 2024-09-18T01:08:32.319 INFO:teuthology.orchestra.run.smithi120.stdout:1591 read 0x7199572 thru 0x719ef04 (0x5993 bytes) 2024-09-18T01:08:32.319 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.328 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:32.326+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:32.330 INFO:teuthology.orchestra.run.smithi120.stdout:1593 write 0xe83261c thru 0xe839ce7 (0x76cc bytes) 2024-09-18T01:08:32.331 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.331 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.334 INFO:teuthology.orchestra.run.smithi120.stdout:1594 write 0x80b73db thru 0x80c66f9 (0xf31f bytes) 2024-09-18T01:08:32.340 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:32.353 INFO:teuthology.orchestra.run.smithi120.stdout:1597 trunc from 0xe839ce8 to 0x81742d0 2024-09-18T01:08:32.378 INFO:teuthology.orchestra.run.smithi120.stdout:1599 read 0x54a6101 thru 0x54a8820 (0x2720 bytes) 2024-09-18T01:08:32.378 INFO:teuthology.orchestra.run.smithi120.stdout:1602 read 0x40ed0b8 thru 0x40ed544 (0x48d bytes) 2024-09-18T01:08:32.378 INFO:teuthology.orchestra.run.smithi120.stdout:1603 read 0x6efcacc thru 0x6efce2a (0x35f bytes) 2024-09-18T01:08:32.378 INFO:teuthology.orchestra.run.smithi120.stdout:1604 punch from 0x28dfd51 to 0x28e3feb, (0x429a bytes) 2024-09-18T01:08:32.379 INFO:teuthology.orchestra.run.smithi120.stdout:1605 read 0x6983af5 thru 0x69905d6 (0xcae2 bytes) 2024-09-18T01:08:32.379 INFO:teuthology.orchestra.run.smithi120.stdout:1606 punch from 0x4d33a16 to 0x4d3dd84, (0xa36e bytes) 2024-09-18T01:08:32.388 INFO:teuthology.orchestra.run.smithi120.stdout:1608 write 0x9c88699 thru 0x9c88cb9 (0x621 bytes) 2024-09-18T01:08:32.392 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.392 INFO:teuthology.orchestra.run.smithi120.stdout:1611 read 0x2f75ae1 thru 0x2f8429e (0xe7be bytes) 2024-09-18T01:08:32.392 INFO:teuthology.orchestra.run.smithi120.stdout:1612 read 0x8761433 thru 0x876acd9 (0x98a7 bytes) 2024-09-18T01:08:32.403 INFO:teuthology.orchestra.run.smithi120.stdout:1614 writesame 0xd675b52 thru 0xd684451 data_size 0x80(0xe900 bytes) 2024-09-18T01:08:32.403 INFO:teuthology.orchestra.run.smithi120.stdout:1615 read 0x987e9ef thru 0x988599d (0x6faf bytes) 2024-09-18T01:08:32.403 INFO:teuthology.orchestra.run.smithi120.stdout:1616 write 0x2546b00 thru 0x2549fe3 (0x34e4 bytes) 2024-09-18T01:08:32.412 INFO:teuthology.orchestra.run.smithi120.stdout:1617 writesame 0xd7cb8a9 thru 0xd7ceae1 data_size 0x1(0x3239 bytes) 2024-09-18T01:08:32.412 INFO:teuthology.orchestra.run.smithi120.stdout:1619 read 0x2a8265 thru 0x2ae8af (0x664b bytes) 2024-09-18T01:08:32.412 INFO:teuthology.orchestra.run.smithi120.stdout:1620 read 0x7cc32ad thru 0x7ccd84a (0xa59e bytes) 2024-09-18T01:08:32.412 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.412 INFO:teuthology.orchestra.run.smithi120.stdout:1623 read 0xbdb3ec6 thru 0xbdb991d (0x5a58 bytes) 2024-09-18T01:08:32.413 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.413 INFO:teuthology.orchestra.run.smithi120.stdout:1625 write 0xa745f4 thru 0xa78ae5 (0x44f2 bytes) 2024-09-18T01:08:32.414 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.414 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.415 INFO:teuthology.orchestra.run.smithi120.stdout:1628 trunc from 0xd7ceae2 to 0xc3f67fd 2024-09-18T01:08:32.420 INFO:teuthology.orchestra.run.smithi120.stdout:1629 write 0xaf7cf5b thru 0xaf84be1 (0x7c87 bytes) 2024-09-18T01:08:32.429 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:32.427+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:32.438 INFO:teuthology.orchestra.run.smithi120.stdout:1631 trunc from 0xc3f67fd to 0x3a35eab 2024-09-18T01:08:32.451 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.451 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.454 INFO:teuthology.orchestra.run.smithi120.stdout:1632 trunc from 0x3a35eab to 0x1f3a13a 2024-09-18T01:08:32.461 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:32.483 INFO:teuthology.orchestra.run.smithi120.stdout:1637 write 0xd1738a3 thru 0xd17b7c9 (0x7f27 bytes) 2024-09-18T01:08:32.486 INFO:teuthology.orchestra.run.smithi120.stdout:1638 write 0x2fe91ad thru 0x2ff8ed1 (0xfd25 bytes) 2024-09-18T01:08:32.489 INFO:teuthology.orchestra.run.smithi120.stdout:1639 write 0x452b52 thru 0x45ab08 (0x7fb7 bytes) 2024-09-18T01:08:32.493 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.493 INFO:teuthology.orchestra.run.smithi120.stdout:1642 writesame 0x810310e thru 0x8111963 data_size 0x3(0xe856 bytes) 2024-09-18T01:08:32.505 INFO:teuthology.orchestra.run.smithi120.stdout:1643 write 0x9e8e1ab thru 0x9e9b625 (0xd47b bytes) 2024-09-18T01:08:32.510 INFO:teuthology.orchestra.run.smithi120.stdout:1644 read 0x40b36c3 thru 0x40c2791 (0xf0cf bytes) 2024-09-18T01:08:32.510 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.510 INFO:teuthology.orchestra.run.smithi120.stdout:1647 writesame 0x647bd3e thru 0x648b33b data_size 0x2e(0xf5fe bytes) 2024-09-18T01:08:32.515 INFO:teuthology.orchestra.run.smithi120.stdout:1648 write 0x5bfe23 thru 0x5cacf1 (0xaecf bytes) 2024-09-18T01:08:32.520 INFO:teuthology.orchestra.run.smithi120.stdout:1649 read 0x93c7634 thru 0x93c867d (0x104a bytes) 2024-09-18T01:08:32.520 INFO:teuthology.orchestra.run.smithi120.stdout:1650 punch from 0x3bf920c to 0x3c04c47, (0xba3b bytes) 2024-09-18T01:08:32.520 INFO:teuthology.orchestra.run.smithi120.stdout:1651 write 0xce28b49 thru 0xce2b465 (0x291d bytes) 2024-09-18T01:08:32.523 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.523 INFO:teuthology.orchestra.run.smithi120.stdout:1656 write 0x1f66de thru 0x204b45 (0xe468 bytes) 2024-09-18T01:08:32.525 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.525 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.525 INFO:teuthology.orchestra.run.smithi120.stdout:1662 punch from 0x13c7a4b to 0x13d645c, (0xea11 bytes) 2024-09-18T01:08:32.525 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.525 INFO:teuthology.orchestra.run.smithi120.stdout:1665 read 0x73db5de thru 0x73e5117 (0x9b3a bytes) 2024-09-18T01:08:32.528 INFO:teuthology.orchestra.run.smithi120.stdout:1666 trunc from 0xd17b7ca to 0xa87ac4d 2024-09-18T01:08:32.529 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:32.527+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:32.537 INFO:teuthology.orchestra.run.smithi120.stdout:1668 punch from 0x55f9109 to 0x55fa979, (0x1870 bytes) 2024-09-18T01:08:32.547 INFO:teuthology.orchestra.run.smithi120.stdout:1669 writesame 0xd9b27f3 thru 0xd9b4aa3 data_size 0x1(0x22b1 bytes) 2024-09-18T01:08:32.547 INFO:teuthology.orchestra.run.smithi120.stdout:1670 read 0xa54011a thru 0xa54fca8 (0xfb8f bytes) 2024-09-18T01:08:32.547 INFO:teuthology.orchestra.run.smithi120.stdout:1671 read 0xbb45116 thru 0xbb463de (0x12c9 bytes) 2024-09-18T01:08:32.547 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.558 INFO:teuthology.orchestra.run.smithi120.stdout:1673 trunc from 0xd9b4aa4 to 0x3554bbc 2024-09-18T01:08:32.566 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:08:32.570 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.571 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.571 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:32.583 INFO:teuthology.orchestra.run.smithi120.stdout:1675 trunc from 0x3554bbc to 0xea6630d 2024-09-18T01:08:32.589 INFO:teuthology.orchestra.run.smithi120.stdout:1676 writesame 0x9e59ebb thru 0x9e5bd0a data_size 0x28(0x1e50 bytes) 2024-09-18T01:08:32.592 INFO:teuthology.orchestra.run.smithi120.stdout:1678 read 0xdf21970 thru 0xdf24be1 (0x3272 bytes) 2024-09-18T01:08:32.592 INFO:teuthology.orchestra.run.smithi120.stdout:1679 writesame 0xe333d30 thru 0xe33dfe4 data_size 0x1(0xa2b5 bytes) 2024-09-18T01:08:32.612 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.612 INFO:teuthology.orchestra.run.smithi120.stdout:1681 read 0xadbfd9c thru 0xadc9934 (0x9b99 bytes) 2024-09-18T01:08:32.612 INFO:teuthology.orchestra.run.smithi120.stdout:1682 write 0x31b4bd thru 0x325fdd (0xab21 bytes) 2024-09-18T01:08:32.615 INFO:teuthology.orchestra.run.smithi120.stdout:1683 read 0x6dfae66 thru 0x6e05456 (0xa5f1 bytes) 2024-09-18T01:08:32.615 INFO:teuthology.orchestra.run.smithi120.stdout:1684 read 0x65095c8 thru 0x6515113 (0xbb4c bytes) 2024-09-18T01:08:32.616 INFO:teuthology.orchestra.run.smithi120.stdout:1687 trunc from 0xea6630d to 0xe45eb98 2024-09-18T01:08:32.629 INFO:teuthology.orchestra.run.smithi120.stdout:1688 trunc from 0xe45eb98 to 0x81474a4 2024-09-18T01:08:32.629 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:32.629+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:32.655 INFO:teuthology.orchestra.run.smithi120.stdout:1689 write 0xd874568 thru 0xd87ee64 (0xa8fd bytes) 2024-09-18T01:08:32.658 INFO:teuthology.orchestra.run.smithi120.stdout:1690 read 0x823b9f7 thru 0x824694f (0xaf59 bytes) 2024-09-18T01:08:32.672 INFO:teuthology.orchestra.run.smithi120.stdout:1691 trunc from 0xd87ee65 to 0x395799a 2024-09-18T01:08:32.682 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:32.682 INFO:teuthology.orchestra.run.smithi120.stdout:1697 punch from 0x9f252c to 0x9fa15c, (0x7c30 bytes) 2024-09-18T01:08:32.682 INFO:teuthology.orchestra.run.smithi120.stdout:1699 read 0x134cbc thru 0x13e632 (0x9977 bytes) 2024-09-18T01:08:32.683 INFO:teuthology.orchestra.run.smithi120.stdout:1700 clone 19 order 25 su 262144 sc 6 2024-09-18T01:08:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.698 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:32.739 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:32.738+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:32.808 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.808 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:32.860 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:32.858+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:32.923 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:32.923 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:32.932 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:32.932 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:32.931+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:33.033 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:33.031+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:33.074 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:33.074 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:33.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:33.167 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:33.165+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:33.186 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:33.228 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:33.228 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:33.234 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:33.232+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:33.237 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:33.309 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:33.309 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:33.326 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:33.326 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:33.335 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:33.334+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:33.383 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:33.383 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:33.391 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:33.435 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:33.435+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:33.536 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:33.534+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:33.537 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:33.537 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:33.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:33.537 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:33.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356559373000003, 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.538 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:33.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356542804999997, 2024-09-18T01:08:33.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:33.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356526619, 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356515049999999, 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356505683999998, 2024-09-18T01:08:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356493501000003, 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356477806000001, 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356460652000003, 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356438486000002, 2024-09-18T01:08:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356422035999998, 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356412581999997, 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356378530999997, 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356358806999999, 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.35634057, 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:33.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356319995, 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356300767999997, 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.356281414999998, 2024-09-18T01:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.35626276, 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.347480810999997, 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:50519 3.19 3:9a8c2274:::rbd_data.11761711706f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.1160297020000005, 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026218866, 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 50519 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418749+0000", 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.85e-07 2024-09-18T01:08:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418754+0000", 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2910000000000001e-06 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418764+0000", 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418769+0000", 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1789999999999999e-06 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418814+0000", 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4350000000000001e-05 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418863+0000", 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9264000000000001e-05 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.423079+0000", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042156269999999996 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444883+0000", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021804975000000001 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444896+0000", 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2344000000000001e-05 2024-09-18T01:08:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444929+0000", 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.269e-05 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444967+0000", 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8818e-05 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:33.637 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:33.637 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:33.638 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:33.639 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:33.639 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:33.639 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:33.640 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:33.640 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:33.640 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:33.640 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:33.641 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:33.641 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:33.641 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:33.642 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:33.642 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:33.637+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:33.739 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:33.738+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:33.783 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:33.784 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:33.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:33.808 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:08:33.840 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:33.839+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:33.920 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:33.920 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:33.920 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:33.920 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:33.920 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:33.929 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:33.973 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:33.971+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:34.033 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.092148385000002, 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:34.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.038 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.964220345000001, 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:34.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.913846284999998, 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:34.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.823953068999998, 2024-09-18T01:08:34.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:34.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.052 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.822357320999998, 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:34.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.821546701999999, 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:34.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.438173890999998, 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:34.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.874573834000003, 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.856812748000003, 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:34.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.854156045000003, 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:34.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.271508502000003, 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:34.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.083 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.857195042999997, 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:34.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.355496021, 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:34.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.353919134000002, 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.324171543999999, 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:34.097 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:34.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.278528383000001, 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.168792058000001, 2024-09-18T01:08:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.165138628000001, 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:49241 3.9 e73/23 3:95d710ee:::rbd_data.117667234fca.0000000000000001:head v 73'60, mlcod=64'58)", 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.454018563, 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034975370000000002, 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.577855+0000", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579263+0000", 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014074720000000001 2024-09-18T01:08:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579278+0000", 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5509999999999999e-05 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579288+0000", 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8490000000000001e-06 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579364+0000", 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5507999999999996e-05 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.579389+0000", 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5196999999999999e-05 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.581309+0000", 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001919802 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:20.581352+0000", 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3738e-05 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:52190 3.4 e82/23 3:2587a2b7:::rbd_data.11762afc950e.0000000000000022:head v 82'135, mlcod=77'133)", 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4289224379999999, 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038334850000000002, 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602975+0000", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3629999999999999e-05 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602984+0000", 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0820000000000005e-06 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602992+0000", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6920000000000002e-06 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603100+0000", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107977 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603124+0000", 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4544000000000001e-05 2024-09-18T01:08:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606735+0000", 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036103580000000001 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606784+0000", 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9697000000000002e-05 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.125 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:34.126 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:34.127 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:34.125+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:34.227 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:34.226+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:34.268 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:34.268 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:34.268 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:34.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:34.328 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:34.326+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:34.380 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:34.380 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:34.380 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:34.380 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:34.380 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:34.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:34.429 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:34.428+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:34.530 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:34.528+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38805 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2424832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961304395999999, 2024-09-18T01:08:34.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244350999999999, 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38805 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569048+0000", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569057+0000", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4530000000000008e-06 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569060+0000", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0170000000000001e-06 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611025+0000", 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041964317000000001 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611073+0000", 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8037000000000001e-05 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611288+0000", 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215596 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38806 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961284571, 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042508381999999997, 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38806 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4400000000000001e-07 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569065+0000", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.206e-06 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569068+0000", 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2890000000000002e-06 2024-09-18T01:08:34.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569071+0000", 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611316+0000", 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244918999999999 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611360+0000", 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4239000000000001e-05 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611572+0000", 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212186 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.541 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38807 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961274818, 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042802420000000001, 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38807 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569074+0000", 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:34.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569075+0000", 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569078+0000", 2024-09-18T01:08:34.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9749999999999999e-06 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569080+0000", 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611602+0000", 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042521681999999998 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611650+0000", 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8179000000000001e-05 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611876+0000", 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022609299999999999 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38808 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961264968999998, 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043110682999999997, 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38808 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569084+0000", 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:34.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569085+0000", 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569088+0000", 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569090+0000", 2024-09-18T01:08:34.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3120000000000001e-06 2024-09-18T01:08:34.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611901+0000", 2024-09-18T01:08:34.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042810991999999999 2024-09-18T01:08:34.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611955+0000", 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3983000000000001e-05 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612194+0000", 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239182 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38809 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 475136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961246643999999, 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043389637000000002, 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38809 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.553 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:34.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569122+0000", 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0254999999999998e-05 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569127+0000", 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569130+0000", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.016e-06 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612224+0000", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043094406000000002 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612275+0000", 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1054000000000003e-05 2024-09-18T01:08:34.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612491+0000", 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215987 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38810 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961213797999999, 2024-09-18T01:08:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043652712000000003, 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38810 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569135+0000", 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569136+0000", 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.083e-06 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569141+0000", 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569143+0000", 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3369999999999998e-06 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612521+0000", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043377538 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612566+0000", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5485000000000002e-05 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612787+0000", 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221158 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38811 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961202517, 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043930856999999997, 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38811 2024-09-18T01:08:34.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569147+0000", 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0449999999999999e-06 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569150+0000", 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569152+0000", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612830+0000", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043677597999999998 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612880+0000", 2024-09-18T01:08:34.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0219999999999997e-05 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613077+0000", 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019646600000000001 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38812 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961193307999999, 2024-09-18T01:08:34.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044183260000000002, 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38812 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:34.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569156+0000", 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999991e-07 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569160+0000", 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3759999999999999e-06 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569162+0000", 2024-09-18T01:08:34.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613107+0000", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043945166000000001 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613153+0000", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613338+0000", 2024-09-18T01:08:34.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018564800000000001 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38813 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1130496~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.96118384, 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044519186000000002, 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38813 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:34.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569165+0000", 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569166+0000", 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2399999999999996e-07 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569168+0000", 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9229999999999998e-06 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569178+0000", 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6479999999999998e-06 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613370+0000", 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044191800000000003 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613416+0000", 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6431999999999999e-05 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613684+0000", 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026725300000000001 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.576 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38814 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961167009, 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044848796000000003, 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38814 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569183+0000", 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.096e-06 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569187+0000", 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569191+0000", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1930000000000002e-06 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613735+0000", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044543886999999997 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613784+0000", 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9858999999999998e-05 2024-09-18T01:08:34.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614030+0000", 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024569700000000002 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38815 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1458176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961154898, 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045101439, 2024-09-18T01:08:34.581 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38815 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569194+0000", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569195+0000", 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0389999999999999e-06 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569198+0000", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9210000000000001e-06 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569200+0000", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.367e-06 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614061+0000", 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044861375000000002 2024-09-18T01:08:34.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614104+0000", 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3142000000000001e-05 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614295+0000", 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190383 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38816 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1622016~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961145788, 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045464629999999999, 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38816 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569203+0000", 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569206+0000", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.754e-06 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569211+0000", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.109e-06 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569213+0000", 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.379e-06 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614331+0000", 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045118395999999998 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614380+0000", 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614667+0000", 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028724000000000002 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38817 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1785856~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961132395, 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045799302, 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38817 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569217+0000", 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.035e-06 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569221+0000", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.839e-06 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569223+0000", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3300000000000001e-06 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614711+0000", 2024-09-18T01:08:34.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045487811000000003 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614772+0000", 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0729000000000001e-05 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615015+0000", 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024334399999999999 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38818 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961122345, 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046120525000000002, 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.595 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38818 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569230+0000", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0260000000000001e-06 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569234+0000", 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:08:34.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569237+0000", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3750000000000001e-06 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615046+0000", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045809435000000003 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615096+0000", 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9654999999999998e-05 2024-09-18T01:08:34.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615346+0000", 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025073700000000001 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38819 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2113536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961106558000001, 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046434472999999997, 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38819 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569243+0000", 2024-09-18T01:08:34.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.291e-06 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569247+0000", 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.031e-06 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569250+0000", 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615395+0000", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046145046000000002 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615437+0000", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2407999999999998e-05 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615676+0000", 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239137 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.603 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38820 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2277376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961095660000002, 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046745223000000002, 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38820 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569254+0000", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-07 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569258+0000", 2024-09-18T01:08:34.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6489999999999999e-06 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569261+0000", 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.322e-06 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615704+0000", 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046442986999999998 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615764+0000", 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9871000000000003e-05 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615998+0000", 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023420099999999999 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38821 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2441216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.608 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961084807999999, 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047015754999999999, 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38821 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569264+0000", 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569265+0000", 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8599999999999996e-07 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569269+0000", 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8800000000000001e-06 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569271+0000", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616017+0000", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046746593000000003 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616061+0000", 2024-09-18T01:08:34.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3894999999999999e-05 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616279+0000", 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000217875 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38822 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.961074736, 2024-09-18T01:08:34.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047344570000000002, 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38822 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569275+0000", 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.026e-06 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569278+0000", 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999999e-06 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569284+0000", 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.071e-06 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616318+0000", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047033859999999997 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616374+0000", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5636000000000001e-05 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616618+0000", 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024412699999999999 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38823 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.957098163000001, 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045935959999999998, 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38823 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573251+0000", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573255+0000", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2520000000000001e-06 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573266+0000", 2024-09-18T01:08:34.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0958000000000001e-05 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573271+0000", 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1229999999999999e-06 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616299+0000", 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043027351999999998 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616351+0000", 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2284000000000003e-05 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616681+0000", 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033019400000000002 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619078+0000", 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023970749999999998 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619093+0000", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4338999999999999e-05 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619143+0000", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0346000000000002e-05 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619186+0000", 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3226999999999998e-05 2024-09-18T01:08:34.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38824 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.951758814000002, 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042771339999999998, 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:34.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38824 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:34.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578590+0000", 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578594+0000", 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0609999999999997e-06 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:34.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578605+0000", 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0477e-05 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578611+0000", 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9540000000000001e-06 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617084+0000", 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.038473680000000003 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617135+0000", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0139000000000002e-05 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617316+0000", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181082 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621302+0000", 2024-09-18T01:08:34.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039860210000000002 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621309+0000", 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6490000000000008e-06 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621336+0000", 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6828000000000001e-05 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621361+0000", 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4621999999999999e-05 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.631 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:34.631 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:34.631 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:34.631 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:08:34.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:08:34.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:34.634 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:34.632+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:34.734 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:34.733+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:34.750 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:34.751 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:34.759 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:34.845 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:34.844+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:34.887 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-18T01:08:34.906 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:34.907 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:34.907 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:34.907 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:34.907 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:34.915 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:34.936 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:34.934+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 87, 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:08:34.806720+0000", 2024-09-18T01:08:34.966 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:07:01.953049+0000", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5804032, 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:34.967 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "jewel", 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:08:34.968 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:34.969 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:08:34.970 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:34.971 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:34.972 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.973 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:08:34.974 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:08:34.975 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:34.976 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:34.977 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:34.978 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:34.979 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:34.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:34.981 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:08:34.982 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:34.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:08:34.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "87", 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:34.985 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 41, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 87, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:34.986 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:34.987 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:34.988 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:34.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:08:34.994 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:34.995 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:34.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.000 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 18, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.001 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.005 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:35.006 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.009 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:08:35.011 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.012 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.018 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.019 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.024 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 82, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.025 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.026 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:35.030 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.031 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.032 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:35.034 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.035 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.036 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.037 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.039 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:35.041 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [], 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:08:35.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 37, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 39, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:35.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 41, 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 41, 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:35.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:35.045 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:35.048 INFO:tasks.thrashosds.thrasher:j is {'epoch': 87, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:08:34.806720+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:07:01.953049+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}, {'begin': 39, 'length': 1}, {'begin': 41, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:08:35.048 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:08:35.048 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:35.045+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:35.060 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:35.061 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.467251931, 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:35.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:35.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.458920259000003, 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.066 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:35.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.070 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.455579405000002, 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:35.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:35.075 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.428124576000002, 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:35.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.394159258999998, 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.080 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.313191672999999, 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.085 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:35.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:35.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.313169834999997, 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.089 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:35.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:35.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.422824534999997, 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.094 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:35.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:35.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.421281669999999, 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.099 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.421260513, 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:35.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.421241698999999, 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:35.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:35.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.967734903, 2024-09-18T01:08:35.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:35.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:08:35.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:08:35.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.967723522999997, 2024-09-18T01:08:35.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:35.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:35.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:35.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.967709603000003, 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:35.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:35.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:35.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.491175018, 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:08:35.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:08:35.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:08:35.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.491163538999999, 2024-09-18T01:08:35.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:35.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:35.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:35.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.33597168, 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.398965681, 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.335015167999998, 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.197271566000001, 2024-09-18T01:08:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.169 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:35.171 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:35.172 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:35.170+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1702 punch from 0xf203e8 to 0xf2df50, (0xdb68 bytes) 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1704 trunc from 0x395799a to 0xd8c74de 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1706 trunc from 0xd8c74de to 0xd50d78a 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1707 read 0xf389ce thru 0xf3e6be (0x5cf1 bytes) 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1709 read 0x3fa2aac thru 0x3fa5d4c (0x32a1 bytes) 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1710 write 0x7b686ba thru 0x7b6ddc7 (0x570e bytes) 2024-09-18T01:08:35.172 INFO:teuthology.orchestra.run.smithi120.stdout:1711 trunc from 0xd50d78a to 0x6050fc7 2024-09-18T01:08:35.183 INFO:teuthology.orchestra.run.smithi120.stdout:1712 writesame 0xec1b1b3 thru 0xec298ac data_size 0xa2(0xe6fa bytes) 2024-09-18T01:08:35.183 INFO:teuthology.orchestra.run.smithi120.stdout:1714 write 0x357ec45 thru 0x3586610 (0x79cc bytes) 2024-09-18T01:08:35.201 INFO:teuthology.orchestra.run.smithi120.stdout:1715 trunc from 0xec298ad to 0x7b5943 2024-09-18T01:08:35.210 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:35.227 INFO:teuthology.orchestra.run.smithi120.stdout:1721 write 0xa091157 thru 0xa0964d2 (0x537c bytes) 2024-09-18T01:08:35.229 INFO:teuthology.orchestra.run.smithi120.stdout:1724 read 0x93d9b8e thru 0x93e68d2 (0xcd45 bytes) 2024-09-18T01:08:35.229 INFO:teuthology.orchestra.run.smithi120.stdout:1727 write 0x7df7356 thru 0x7dfe446 (0x70f1 bytes) 2024-09-18T01:08:35.231 INFO:teuthology.orchestra.run.smithi120.stdout:1728 read 0x62ddb80 thru 0x62de5fc (0xa7d bytes) 2024-09-18T01:08:35.231 INFO:teuthology.orchestra.run.smithi120.stdout:1729 writesame 0x9985c65 thru 0x998de77 data_size 0x43(0x8213 bytes) 2024-09-18T01:08:35.238 INFO:teuthology.orchestra.run.smithi120.stdout:1730 trunc from 0xa0964d3 to 0xeaaacf8 2024-09-18T01:08:35.244 INFO:teuthology.orchestra.run.smithi120.stdout:1731 read 0x119c1bb thru 0x11a02cd (0x4113 bytes) 2024-09-18T01:08:35.244 INFO:teuthology.orchestra.run.smithi120.stdout:1732 read 0xc5e0d69 thru 0xc5e4e43 (0x40db bytes) 2024-09-18T01:08:35.244 INFO:teuthology.orchestra.run.smithi120.stdout:1733 writesame 0xe955a38 thru 0xe9590c9 data_size 0x16(0x3692 bytes) 2024-09-18T01:08:35.254 INFO:teuthology.orchestra.run.smithi120.stdout:1734 trunc from 0xeaaacf8 to 0x87d4e84 2024-09-18T01:08:35.267 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:35.267 INFO:teuthology.orchestra.run.smithi120.stdout:1736 write 0x28e555e thru 0x28e94cd (0x3f70 bytes) 2024-09-18T01:08:35.269 INFO:teuthology.orchestra.run.smithi120.stdout:1737 write 0x7cfa9c0 thru 0x7d0530a (0xa94b bytes) 2024-09-18T01:08:35.271 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:35.270+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:35.271 INFO:teuthology.orchestra.run.smithi120.stdout:1738 read 0x817faad thru 0x81839cf (0x3f23 bytes) 2024-09-18T01:08:35.272 INFO:teuthology.orchestra.run.smithi120.stdout:1739 punch from 0x1a5727d to 0x1a63f29, (0xccac bytes) 2024-09-18T01:08:35.272 INFO:teuthology.orchestra.run.smithi120.stdout:1741 write 0x718f3a0 thru 0x719e145 (0xeda6 bytes) 2024-09-18T01:08:35.284 INFO:teuthology.orchestra.run.smithi120.stdout:1742 write 0xc8402a9 thru 0xc8421c5 (0x1f1d bytes) 2024-09-18T01:08:35.287 INFO:teuthology.orchestra.run.smithi120.stdout:1744 read 0x49d5a57 thru 0x49db089 (0x5633 bytes) 2024-09-18T01:08:35.288 INFO:teuthology.orchestra.run.smithi120.stdout:1745 write 0x26f58a1 thru 0x2703b25 (0xe285 bytes) 2024-09-18T01:08:35.290 INFO:teuthology.orchestra.run.smithi120.stdout:1747 read 0x25022cc thru 0x2503055 (0xd8a bytes) 2024-09-18T01:08:35.290 INFO:teuthology.orchestra.run.smithi120.stdout:1748 writesame 0x20a049f thru 0x20aa906 data_size 0x4(0xa468 bytes) 2024-09-18T01:08:35.295 INFO:teuthology.orchestra.run.smithi120.stdout:1750 read 0x55d9033 thru 0x55e3ff8 (0xafc6 bytes) 2024-09-18T01:08:35.310 INFO:teuthology.orchestra.run.smithi120.stdout:1751 trunc from 0xc8421c6 to 0xf31ab7 2024-09-18T01:08:35.321 INFO:teuthology.orchestra.run.smithi120.stdout:1752 punch from 0x8919fd to 0x89ecd2, (0xd2d5 bytes) 2024-09-18T01:08:35.323 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:35.324 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:35.324 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:35.324 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:35.333 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:35.337 INFO:teuthology.orchestra.run.smithi120.stdout:1753 write 0x987004b thru 0x987f407 (0xf3bd bytes) 2024-09-18T01:08:35.339 INFO:teuthology.orchestra.run.smithi120.stdout:1754 read 0xcb4612 thru 0xcbb1cd (0x6bbc bytes) 2024-09-18T01:08:35.339 INFO:teuthology.orchestra.run.smithi120.stdout:1755 writesame 0x1b0a116 thru 0x1b0aacc data_size 0x1(0x9b7 bytes) 2024-09-18T01:08:35.344 INFO:teuthology.orchestra.run.smithi120.stdout:1757 write 0x87c4cd thru 0x889dd8 (0xd90c bytes) 2024-09-18T01:08:35.345 INFO:teuthology.orchestra.run.smithi120.stdout:1758 punch from 0x8bd4b3d to 0x8bd886f, (0x3d32 bytes) 2024-09-18T01:08:35.345 INFO:teuthology.orchestra.run.smithi120.stdout:1759 writesame 0x6d023cf thru 0x6d0c48a data_size 0x7f(0xa0bc bytes) 2024-09-18T01:08:35.348 INFO:teuthology.orchestra.run.smithi120.stdout:1760 read 0xc94078 thru 0xca2cff (0xec88 bytes) 2024-09-18T01:08:35.348 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:35.349 INFO:teuthology.orchestra.run.smithi120.stdout:1763 clone 20 order 20 su 524288 sc 7 2024-09-18T01:08:35.374 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:35.373+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:35.474 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:35.473+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:35.476 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:35.476 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:35.476 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:35.476 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:35.476 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:35.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:35.575 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:35.574+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:35.590 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564859761999998, 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564817611000002, 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564784183, 2024-09-18T01:08:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564746683000003, 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564725105999997, 2024-09-18T01:08:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564705965999998, 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564685574999999, 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564656175000003, 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564627385000001, 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564606689999998, 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564587205999999, 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564553295000003, 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.564517295999998, 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.480069323999999, 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.480059115000003, 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.979657889000002, 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.964171720000003, 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:35.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:35.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.956280999999997, 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.686 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:35.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:35.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.930211997000001, 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:35.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:35.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.925043205000001, 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:35.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:35.701 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:35.703 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:35.701+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:35.703 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:35.803 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:35.801+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:35.823 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:35.824 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:35.826 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:08:35.845 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:35.845 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:35.854 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:35.855 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:35.865 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:35.866 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:35.882 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:35.899 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:35.899 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:35.904 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:35.902+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:35.915 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:35.915 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:35.933 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:35.933 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:35.949 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:35.950 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:35.971 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:35.971 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:35.981 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:36.005 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:36.004+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:36.083 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.084 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.092 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:36.137 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:36.136+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:36.196 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.196 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.204 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:36.246 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:36.245+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:36.309 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:36.307+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:36.349 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.349 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.358 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:36.408 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:36.408+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:36.461 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.461 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:36.509 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:36.508+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:36.575 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.576 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:36.610 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:36.609+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:36.686 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.686 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.694 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:36.738 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:36.737+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:36.797 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.797 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:36.829 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:08:36.829 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:36.828+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:36.913 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:36.912+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:36.948 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:36.949 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:36.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:37.014 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:37.013+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:37.064 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.064 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.072 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:37.114 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:37.113+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:37.214 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.215 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.215 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:37.214+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:37.223 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:37.316 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:37.315+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:37.367 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.368 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:37.418 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:37.416+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:37.518 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:37.517+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:37.520 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.520 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.529 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:37.619 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:37.618+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:37.632 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.632 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:37.720 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:37.719+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:37.785 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.785 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:37.821 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:37.820+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:37.904 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:37.904 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:37.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:37.947 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-18T01:08:37.947 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:37.946+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:37.962 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:38.018 INFO:teuthology.orchestra.run.smithi120.stdout:1764 read 0x68d777f thru 0x68d7f81 (0x803 bytes) 2024-09-18T01:08:38.018 INFO:teuthology.orchestra.run.smithi120.stdout:1765 punch from 0x8c17948 to 0x8c25625, (0xdcdd bytes) 2024-09-18T01:08:38.021 INFO:teuthology.orchestra.run.smithi120.stdout:1768 punch from 0x8b49884 to 0x8b5167a, (0x7df6 bytes) 2024-09-18T01:08:38.022 INFO:teuthology.orchestra.run.smithi120.stdout:1769 write 0x771d522 thru 0x771e106 (0xbe5 bytes) 2024-09-18T01:08:38.023 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:38.023+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:38.028 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:38.029 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:38.032 INFO:teuthology.orchestra.run.smithi120.stdout:1770 trunc from 0x987f408 to 0x308a838 2024-09-18T01:08:38.043 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:38.076 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:38.091 INFO:teuthology.orchestra.run.smithi120.stdout:1774 write 0x944b140 thru 0x944bc81 (0xb42 bytes) 2024-09-18T01:08:38.094 INFO:teuthology.orchestra.run.smithi120.stdout:1777 writesame 0x2b38cec thru 0x2b41347 data_size 0x2(0x865c bytes) 2024-09-18T01:08:38.095 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:38.095 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:38.105 INFO:teuthology.orchestra.run.smithi120.stdout:1778 write 0x22f2a70 thru 0x22f5b3b (0x30cc bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1779 punch from 0x2bf598b to 0x2bff50f, (0x9b84 bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1780 read 0x4f720d8 thru 0x4f76f0e (0x4e37 bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1781 read 0x102b5f7 thru 0x10389f7 (0xd401 bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1783 punch from 0xcbfc34 to 0xccec73, (0xf03f bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1785 punch from 0x81bd13e to 0x81c4ba3, (0x7a65 bytes) 2024-09-18T01:08:38.109 INFO:teuthology.orchestra.run.smithi120.stdout:1786 write 0x81da7ac thru 0x81e7e15 (0xd66a bytes) 2024-09-18T01:08:38.112 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:38.113 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:38.114 INFO:teuthology.orchestra.run.smithi120.stdout:1788 writesame 0x8febfdb thru 0x8ffbb43 data_size 0x1(0xfb69 bytes) 2024-09-18T01:08:38.124 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:38.124+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:38.128 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:38.129 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:38.144 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:38.145 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:38.159 INFO:teuthology.orchestra.run.smithi120.stdout:1789 write 0xae0b0d1 thru 0xae0d87c (0x27ac bytes) 2024-09-18T01:08:38.162 INFO:teuthology.orchestra.run.smithi120.stdout:1790 trunc from 0xae0d87d to 0xae41312 2024-09-18T01:08:38.163 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:38.163 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:38.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:38.178 INFO:teuthology.orchestra.run.smithi120.stdout:1791 trunc from 0xae41312 to 0x17647af 2024-09-18T01:08:38.205 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:38.213 INFO:teuthology.orchestra.run.smithi120.stdout:1793 writesame 0x36efea5 thru 0x36f3455 data_size 0x1(0x35b1 bytes) 2024-09-18T01:08:38.213 INFO:teuthology.orchestra.run.smithi120.stdout:1796 punch from 0x13a60ef to 0x13b208f, (0xbfa0 bytes) 2024-09-18T01:08:38.213 INFO:teuthology.orchestra.run.smithi120.stdout:1797 read 0x4e4d6c thru 0x4e5586 (0x81b bytes) 2024-09-18T01:08:38.213 INFO:teuthology.orchestra.run.smithi120.stdout:1799 writesame 0x233756c thru 0x2342878 data_size 0x21(0xb30d bytes) 2024-09-18T01:08:38.216 INFO:teuthology.orchestra.run.smithi120.stdout:1800 read 0x1ff9323 thru 0x2001bf4 (0x88d2 bytes) 2024-09-18T01:08:38.226 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:38.225+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:38.227 INFO:teuthology.orchestra.run.smithi120.stdout:1801 write 0x7d97e36 thru 0x7d9bea4 (0x406f bytes) 2024-09-18T01:08:38.231 INFO:teuthology.orchestra.run.smithi120.stdout:1802 read 0x52cda62 thru 0x52d1608 (0x3ba7 bytes) 2024-09-18T01:08:38.231 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:38.234 INFO:teuthology.orchestra.run.smithi120.stdout:1804 trunc from 0x7d9bea5 to 0x57e2b16 2024-09-18T01:08:38.245 INFO:teuthology.orchestra.run.smithi120.stdout:1805 punch from 0xd7d4d7 to 0xd81bec, (0x4715 bytes) 2024-09-18T01:08:38.245 INFO:teuthology.orchestra.run.smithi120.stdout:1806 read 0x2a89557 thru 0x2a8c843 (0x32ed bytes) 2024-09-18T01:08:38.253 INFO:teuthology.orchestra.run.smithi120.stdout:1807 write 0x6fb3082 thru 0x6fb7258 (0x41d7 bytes) 2024-09-18T01:08:38.260 INFO:teuthology.orchestra.run.smithi120.stdout:1808 trunc from 0x6fb7259 to 0x4c5e53e 2024-09-18T01:08:38.270 INFO:teuthology.orchestra.run.smithi120.stdout:1811 punch from 0x116513e to 0x1165f09, (0xdcb bytes) 2024-09-18T01:08:38.270 INFO:teuthology.orchestra.run.smithi120.stdout:1812 read 0x2bb3fff thru 0x2bba311 (0x6313 bytes) 2024-09-18T01:08:38.284 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:38.284 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:38.289 INFO:teuthology.orchestra.run.smithi120.stdout:1813 write 0xd66c033 thru 0xd6721c7 (0x6195 bytes) 2024-09-18T01:08:38.294 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:38.294 INFO:teuthology.orchestra.run.smithi120.stdout:1814 write 0x3ff776 thru 0x40c6c9 (0xcf54 bytes) 2024-09-18T01:08:38.296 INFO:teuthology.orchestra.run.smithi120.stdout:1815 read 0x178235 thru 0x17db14 (0x58e0 bytes) 2024-09-18T01:08:38.297 INFO:teuthology.orchestra.run.smithi120.stdout:1818 write 0xbbf35bb thru 0xbc03506 (0xff4c bytes) 2024-09-18T01:08:38.302 INFO:teuthology.orchestra.run.smithi120.stdout:1819 read 0x310f1ce thru 0x3110dbb (0x1bee bytes) 2024-09-18T01:08:38.312 INFO:teuthology.orchestra.run.smithi120.stdout:1820 trunc from 0xd6721c8 to 0x5701616 2024-09-18T01:08:38.324 INFO:teuthology.orchestra.run.smithi120.stdout:1822 read 0x3692546 thru 0x36a2356 (0xfe11 bytes) 2024-09-18T01:08:38.326 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:38.326+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:38.336 INFO:teuthology.orchestra.run.smithi120.stdout:1823 writesame 0xaea422e thru 0xaeafc57 data_size 0x8d(0xba2a bytes) 2024-09-18T01:08:38.337 INFO:teuthology.orchestra.run.smithi120.stdout:1824 read 0x2ad9583 thru 0x2adf062 (0x5ae0 bytes) 2024-09-18T01:08:38.337 INFO:teuthology.orchestra.run.smithi120.stdout:1825 clone 21 order 24 su 2097152 sc 10 2024-09-18T01:08:38.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:38.363 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:38.378 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:38.379 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:38.395 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:38.395 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.808548569000003, 2024-09-18T01:08:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.800216896999999, 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.796876042999997, 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.769421213999998, 2024-09-18T01:08:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.735455897000001, 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.654488311000001, 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.654466472999999, 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.430 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.764121172999999, 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.762578308000002, 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.762557151000003, 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.762538337000002, 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27796 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.309031541000003, 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095768439999999993, 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27796 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090977+0000", 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090978+0000", 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090980+0000", 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.012e-06 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090983+0000", 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090986+0000", 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.571e-06 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094196+0000", 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032102390000000001 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094252+0000", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5668999999999998e-05 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094484+0000", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232294 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100238+0000", 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0057538700000000003 2024-09-18T01:08:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100473+0000", 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000235722 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100515+0000", 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1439000000000001e-05 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100554+0000", 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9412000000000001e-05 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.309020160999999, 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.309006240999999, 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28353 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.832471656000003, 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095608159999999998, 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28353 2024-09-18T01:08:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567537+0000", 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567538+0000", 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8799999999999998e-07 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567539+0000", 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567542+0000", 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5190000000000001e-06 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567545+0000", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.537e-06 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568881+0000", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001336315 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.568937+0000", 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5448999999999997e-05 2024-09-18T01:08:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569202+0000", 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000265307 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574069+0000", 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048664659999999998 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577026+0000", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029573580000000002 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577071+0000", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4920999999999999e-05 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577098+0000", 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7274e-05 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.832460177000002, 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.677268318000003, 2024-09-18T01:08:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.740262319000003, 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.676311806000001, 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.538568204000001, 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.510 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:38.511 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:38.512 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:38.513 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:38.513 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:38.510+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:38.612 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:38.611+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:38.617 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:38.617 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:38.618 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:38.618 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:38.626 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:38.714 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:38.713+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:38.772 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:38.772 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:38.773 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:38.773 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:38.773 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:38.782 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:38.815 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:38.814+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:38.845 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:08:38.893 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868932606000001, 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868890454999999, 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868857026999997, 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868819526999999, 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868797950000001, 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868778810000002, 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868758419000002, 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868729019, 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868700228999998, 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868679534000002, 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:38.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868660050000003, 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868626139, 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.868590140000002, 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.784142168000002, 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:38.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.784131959, 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:38.959 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:38.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.283730732999999, 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:38.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.268244564, 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:38.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:38.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.260353844000001, 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:38.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:38.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.234284840999997, 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:38.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:38.985 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.229116048999998, 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:38.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:38.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:38.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:38.990 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:38.992 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:38.992 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:38.990+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:39.092 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:39.091+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:39.096 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:39.097 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:39.105 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:39.193 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:39.192+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:39.248 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.248 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:39.249 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:39.249 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:39.249 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:39.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:39.298 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:39.297+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:39.394 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:39.393+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.318378354999993, 2024-09-18T01:08:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 63.215064058999999, 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.738387506000002, 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.654455268, 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.652446134999998, 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.421222409999999, 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.894151483999998, 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.894123702999998, 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:39.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.892704610000003, 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.764642418999998, 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.8288525270000004, 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:08:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:08:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.776861384, 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7767845879999999, 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:08:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:08:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:08:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7722794349999997, 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.482 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7720236390000004, 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:08:39.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7679983109999995, 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:08:39.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7678571610000002, 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:08:39.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:08:39.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7676242569999996, 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:08:39.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7650051950000005, 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:08:39.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:08:39.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7426661610000007, 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:08:39.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:08:39.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.519 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:39.520 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:39.521 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:39.519+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:39.545 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:08:39.621 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:39.620+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:39.672 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.672 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:39.673 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:39.673 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:39.681 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:39.723 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:39.722+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:39.824 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:39.823+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:39.825 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:39.825 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:39.834 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:39.925 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:39.924+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:39.938 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:39.938 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:39.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:39.938 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:39.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.038758356000002, 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:39.939 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:39.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:39.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:39.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:39.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.645579057999996, 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:39.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:39.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.278847757999998, 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:39.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:39.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:39.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:39.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:39.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 51.999460609000003, 2024-09-18T01:08:39.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:39.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:39.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.307138885000001, 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:39.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.307123787999998, 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.974 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.307112638, 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:39.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.307103001000002, 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.303190952999998, 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:39.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.239689703000003, 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:39.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:39.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:39.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:40.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43586 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5816320~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541706985000001, 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015218535, 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43586 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2499999999999999e-07 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394861+0000", 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394864+0000", 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394867+0000", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.409942+0000", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015075312 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410015+0000", 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3090000000000007e-05 2024-09-18T01:08:40.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410079+0000", 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3126999999999994e-05 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43587 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5980160~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541697118999998, 2024-09-18T01:08:40.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015359998, 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43587 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:40.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394871+0000", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4200000000000004e-07 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394874+0000", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394876+0000", 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4399999999999999e-06 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410105+0000", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015228759 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410155+0000", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9987000000000001e-05 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410230+0000", 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4697999999999995e-05 2024-09-18T01:08:40.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43588 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6144000~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541687755000002, 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015489108999999999, 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43588 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394880+0000", 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394884+0000", 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6829999999999999e-06 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394888+0000", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3409999999999999e-06 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394890+0000", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.543e-06 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410258+0000", 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015368371 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410303+0000", 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410368+0000", 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5747999999999995e-05 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43589 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6307840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541673187000001, 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015665806000000001, 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43589 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394895+0000", 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0079999999999999e-06 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394904+0000", 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.234e-06 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394909+0000", 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410406+0000", 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015497273000000001 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410486+0000", 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9722999999999995e-05 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410560+0000", 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4008e-05 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43590 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6471680~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541654104999999, 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01587189, 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43590 2024-09-18T01:08:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8500000000000002e-07 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394915+0000", 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3239999999999999e-06 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394918+0000", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8789999999999996e-06 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394921+0000", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410601+0000", 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015679648000000001 2024-09-18T01:08:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410683+0000", 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2826999999999998e-05 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410785+0000", 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101365 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43591 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6635520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541642881000001, 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015982698, 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43591 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394925+0000", 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.006e-06 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394928+0000", 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394931+0000", 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5720000000000001e-06 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410803+0000", 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015871962999999999 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410847+0000", 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3912000000000002e-05 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410907+0000", 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0025999999999998e-05 2024-09-18T01:08:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43592 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6799360~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.541618524, 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016170249000000001, 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43592 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394950+0000", 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.161e-06 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394955+0000", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9089999999999999e-06 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394958+0000", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.593e-06 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410941+0000", 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015983431999999999 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411019+0000", 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8325999999999995e-05 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411119+0000", 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9463999999999995e-05 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50421 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.703626453, 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020565198, 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50421 2024-09-18T01:08:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232946+0000", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3209999999999998e-06 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232956+0000", 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0368e-05 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232963+0000", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2400000000000001e-06 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233001+0000", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7298000000000001e-05 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233052+0000", 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1659000000000001e-05 2024-09-18T01:08:40.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.236268+0000", 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003215894 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253414+0000", 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017146122999999999 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253429+0000", 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4674e-05 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253468+0000", 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.896e-05 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253506+0000", 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7793999999999998e-05 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50514 3.11 3:8aff915a:::rbd_data.11761711706f.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.540775980999999, 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015404816999999999, 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50514 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395792+0000", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6300000000000005e-07 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395796+0000", 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4490000000000001e-06 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395805+0000", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8580000000000006e-06 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395810+0000", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395844+0000", 2024-09-18T01:08:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769000000000001e-05 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395895+0000", 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0930000000000002e-05 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.398279+0000", 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023842569999999999 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411026+0000", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012747 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411078+0000", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2296000000000002e-05 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411143+0000", 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4561999999999996e-05 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411196+0000", 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3152999999999997e-05 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50582 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.412798685, 2024-09-18T01:08:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027666276, 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50582 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523769+0000", 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8599999999999997e-07 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523774+0000", 2024-09-18T01:08:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523783+0000", 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5149999999999995e-06 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523788+0000", 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523838+0000", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9438999999999999e-05 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523930+0000", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1978000000000006e-05 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.528122+0000", 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004192749 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551279+0000", 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023157015999999999 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551339+0000", 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9410000000000002e-05 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551380+0000", 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0927000000000003e-05 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551435+0000", 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4846999999999999e-05 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:40.055 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:40.056 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:40.056 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:40.056 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:08:40.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:08:40.058 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:40.056+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:40.058 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:40.158 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:40.157+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:40.165 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:40.165 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:40.165 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:40.166 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:40.175 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:40.259 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:40.258+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:40.318 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:40.319 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:40.319 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:40.319 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:40.319 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:40.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:40.345 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:40.345 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:40.345 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 92, 2024-09-18T01:08:40.345 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:08:40.345 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:08:39.543963+0000", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:07:01.953049+0000", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5801984, 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.346 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "jewel", 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:40.349 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:40.350 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:40.351 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:40.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:40.353 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:08:40.354 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:40.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:08:40.356 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:40.357 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:40.358 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:40.360 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:08:40.361 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:08:40.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:08:40.363 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.364 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "92", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 45, 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 92, 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:08:40.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:08:40.366 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:08:40.367 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.369 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:08:40.374 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.375 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.377 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:08:40.378 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.381 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.382 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 18, 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:08:40.383 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.387 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:40.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.391 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:08:40.394 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.395 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:08:40.396 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:40.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.405 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.407 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 82, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.411 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:08:40.413 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:08:40.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:08:40.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.420 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.421 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.423 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.424 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:08:40.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:08:40.426 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 41, 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 43, 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [ 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 37, 2024-09-18T01:08:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 39, 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:08:40.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:08:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:08:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:08:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:08:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.430 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:40.432 INFO:tasks.thrashosds.thrasher:j is {'epoch': 92, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:08:39.543963+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:07:01.953049+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 92, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 1}, {'begin': 43, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}, {'begin': 39, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:08:40.432 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:08:40.433 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:40.431+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:40.514 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:40.535 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.946994343, 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.943360589999998, 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.943337022000001, 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.941761620000001, 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.937328528999998, 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.936006352, 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.931952541000001, 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.906264112000002, 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.875725907000003, 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.875236215000001, 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.810876518999997, 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639838887, 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639810121, 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639786873999999, 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639766870999999, 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639748227999998, 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639729685999999, 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639709100000001, 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.639664830000001, 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.638684638000001, 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:40.640 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:40.640 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:40.534+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:40.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:40.642 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:40.643 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:40.642+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:40.662 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:40.662 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:40.678 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-18T01:08:40.679 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:40.697 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:40.697 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:40.715 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:40.715 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:40.733 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:40.734 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:40.743 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:40.742+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:40.757 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:40.757 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:40.759 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:40.759 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:40.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:40.787 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:40.787 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:40.844 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:40.843+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:40.869 INFO:teuthology.orchestra.run.smithi120.stdout:1827 write 0x58847df thru 0x589369d (0xeebf bytes) 2024-09-18T01:08:40.876 INFO:teuthology.orchestra.run.smithi120.stdout:1828 writesame 0x2927dbc thru 0x2928ecd data_size 0x26(0x1112 bytes) 2024-09-18T01:08:40.888 INFO:teuthology.orchestra.run.smithi120.stdout:1830 write 0xecdff3e thru 0xece9056 (0x9119 bytes) 2024-09-18T01:08:40.891 INFO:teuthology.orchestra.run.smithi120.stdout:1831 write 0xc5773b3 thru 0xc57952f (0x217d bytes) 2024-09-18T01:08:40.894 INFO:teuthology.orchestra.run.smithi120.stdout:1832 read 0xea6d02d thru 0xea6f256 (0x222a bytes) 2024-09-18T01:08:40.894 INFO:teuthology.orchestra.run.smithi120.stdout:1833 read 0x46c9fc3 thru 0x46d6a19 (0xca57 bytes) 2024-09-18T01:08:40.894 INFO:teuthology.orchestra.run.smithi120.stdout:1834 punch from 0x93f01d3 to 0x93f7a59, (0x7886 bytes) 2024-09-18T01:08:40.894 INFO:teuthology.orchestra.run.smithi120.stdout:1837 read 0x6631fea thru 0x6638aaa (0x6ac1 bytes) 2024-09-18T01:08:40.898 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:40.898 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:40.909 INFO:teuthology.orchestra.run.smithi120.stdout:1839 trunc from 0xece9057 to 0x21cd405 2024-09-18T01:08:40.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:40.933 INFO:teuthology.orchestra.run.smithi120.stdout:1840 write 0x320d1b5 thru 0x321669e (0x94ea bytes) 2024-09-18T01:08:40.936 INFO:teuthology.orchestra.run.smithi120.stdout:1841 punch from 0x305e80a to 0x306667a, (0x7e70 bytes) 2024-09-18T01:08:40.939 INFO:teuthology.orchestra.run.smithi120.stdout:1844 trunc from 0x321669f to 0xd5d47e 2024-09-18T01:08:40.944 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:40.943+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:40.974 INFO:teuthology.orchestra.run.smithi120.stdout:1846 write 0xc80f1db thru 0xc81bb74 (0xc99a bytes) 2024-09-18T01:08:40.977 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:40.977 INFO:teuthology.orchestra.run.smithi120.stdout:1849 read 0x5983fd8 thru 0x598ebba (0xabe3 bytes) 2024-09-18T01:08:40.977 INFO:teuthology.orchestra.run.smithi120.stdout:1850 write 0x25e1528 thru 0x25e62d5 (0x4dae bytes) 2024-09-18T01:08:40.980 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:40.980 INFO:teuthology.orchestra.run.smithi120.stdout:1852 punch from 0xbb67ba to 0xbb9acb, (0x3311 bytes) 2024-09-18T01:08:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:1854 punch from 0x3c0215e to 0x3c05bee, (0x3a90 bytes) 2024-09-18T01:08:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:1855 read 0x5284b82 thru 0x52912d7 (0xc756 bytes) 2024-09-18T01:08:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:1856 read 0x8628c31 thru 0x862b452 (0x2822 bytes) 2024-09-18T01:08:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:1857 writesame 0x8a1c41 thru 0x8a244b data_size 0x1(0x80b bytes) 2024-09-18T01:08:40.989 INFO:teuthology.orchestra.run.smithi120.stdout:1859 write 0xe05a021 thru 0xe05a547 (0x527 bytes) 2024-09-18T01:08:40.992 INFO:teuthology.orchestra.run.smithi120.stdout:1860 write 0x19937b3 thru 0x199fea9 (0xc6f7 bytes) 2024-09-18T01:08:40.994 INFO:teuthology.orchestra.run.smithi120.stdout:1861 clone 22 order 25 su 16777216 sc 3 2024-09-18T01:08:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.045 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:41.044+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:41.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:41.146 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:41.145+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.205 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:41.247 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:41.246+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:41.307 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.307 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:41.360 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:41.359+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:41.418 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.419 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.427 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:41.468 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:41.467+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:41.550 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:41.549+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:41.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:08:41.573 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.573 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:41.651 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:41.650+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:41.730 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.730 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:41.752 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:41.751+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:41.846 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.847 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.853 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:41.852+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:41.856 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:41.954 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:41.953+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:41.965 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:41.966 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:41.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:42.055 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:42.054+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:42.077 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.077 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:42.156 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:42.154+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:42.188 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.188 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:42.281 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:42.280+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:42.340 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.341 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:42.357 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:42.356+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:42.459 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:42.458+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:42.494 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.494 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:42.586 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:42.585+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:42.645 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.645 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.653 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:42.661 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:42.660+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:42.762 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:42.761+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:42.797 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.797 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:42.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:42.845 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:08:42.864 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:42.862+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:42.916 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:42.916 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:42.925 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:42.937 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:42.953 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:42.953 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:42.963 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:42.963+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:42.969 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:42.969 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:42.985 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:42.985 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:43.001 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:43.002 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:43.019 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:43.019 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:43.035 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:43.035 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1109 3.10 3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head [write 524288~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.953191378, 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064066519999999997, 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1109 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081536+0000", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081537+0000", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.004e-06 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081679+0000", 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141347 2024-09-18T01:08:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081692+0000", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3417e-05 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081700+0000", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2509999999999994e-06 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081755+0000", 2024-09-18T01:08:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4367999999999999e-05 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.081878+0000", 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012353099999999999 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082019+0000", 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141194 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.082868+0000", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084920700000000004 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087863+0000", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049944100000000003 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087906+0000", 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2928e-05 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.087943+0000", 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6995000000000001e-05 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3121 3.a 3:53d0a4fa:::rbd_data.11769efc726f.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.849877082000006, 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070868229999999999, 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3121 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184850+0000", 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184852+0000", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184860+0000", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184878+0000", 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7567999999999998e-05 2024-09-18T01:08:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184885+0000", 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5410000000000003e-06 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.184948+0000", 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3025000000000001e-05 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.185062+0000", 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113495 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.186182+0000", 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001120493 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191828+0000", 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0056454060000000004 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191845+0000", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191881+0000", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5240999999999997e-05 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:36.191937+0000", 2024-09-18T01:08:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6780000000000002e-05 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9172 3.10 3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head [zero 0~524288,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~1048576,write 2097152~1572864 [fadvise_sequential+fadvise_nocache],zero 3670016~114688,write 3784704~16384 [fadvise_sequential+fadvise_nocache],zero 3801088~114688,write 3915776~16384 [fadvise_sequential+fadvise_nocache],zero 3932160~114688,write 4046848~16384 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache],zero 6258688~32768,zero 7602176~213961,write 7816137~48183 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.373200529000002, 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030671232999999999, 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9172 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661527+0000", 2024-09-18T01:08:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.661529+0000", 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673253+0000", 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01172407 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673275+0000", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2059e-05 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673285+0000", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7599999999999997e-06 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673348+0000", 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3324999999999994e-05 2024-09-18T01:08:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673498+0000", 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149944 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.673745+0000", 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024736000000000002 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.685611+0000", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01186574 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692044+0000", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064329460000000002 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692139+0000", 2024-09-18T01:08:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4754999999999997e-05 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692198+0000", 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466999999999998e-05 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12798 3.10 3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head [sparse-read 0~1310720] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.289268290999999, 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012770844, 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12798 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745459+0000", 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745461+0000", 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745469+0000", 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745488+0000", 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8717999999999999e-05 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745497+0000", 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2790000000000004e-06 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745562+0000", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4831000000000003e-05 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745569+0000", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8569999999999997e-06 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753221+0000", 2024-09-18T01:08:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076519420000000001 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753324+0000", 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103285 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758230+0000", 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049062580000000001 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12812 3.10 3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head [sparse-read 0~1309489] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.287259157999998, 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013751429000000001, 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12812 2024-09-18T01:08:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747468+0000", 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747470+0000", 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.483e-06 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747479+0000", 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747492+0000", 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.309e-05 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747502+0000", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747573+0000", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1617000000000002e-05 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.747581+0000", 2024-09-18T01:08:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758514+0000", 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010933059 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758614+0000", 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9995999999999996e-05 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.761220+0000", 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026061449999999998 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:18509 3.a 3:544cbdc9:::rbd_data.11762d95e74a.00000000000001b9:head [set-alloc-hint object_size 524288 write_size 524288,write 127679~3393] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.056035432999998, 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0076896960000000002, 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 18509 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978692+0000", 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978694+0000", 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0250000000000001e-06 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978702+0000", 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.729e-06 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978718+0000", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5994999999999998e-05 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978727+0000", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6339999999999996e-06 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978793+0000", 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5883000000000006e-05 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978886+0000", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2826999999999997e-05 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.978994+0000", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010802 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.985986+0000", 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069920920000000001 2024-09-18T01:08:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986303+0000", 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031700100000000002 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986340+0000", 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6838000000000002e-05 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:53.986382+0000", 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1652e-05 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28268 3.a 3:51051385:::rbd_data.1176e641238f.0000000000000097:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.528964506999998, 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079262789999999996, 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28268 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505763+0000", 2024-09-18T01:08:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505764+0000", 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505766+0000", 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505774+0000", 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6539999999999999e-06 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505780+0000", 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6319999999999996e-06 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506554+0000", 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077401300000000004 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506624+0000", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0665000000000002e-05 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.506730+0000", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010623700000000001 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511209+0000", 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044783879999999998 2024-09-18T01:08:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513615+0000", 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024057459999999998 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513661+0000", 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5892e-05 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513689+0000", 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.879e-05 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28269 3.a 3:53a10ddf:::rbd_data.1176e641238f.000000000000009c:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.528936725999998, 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079666070000000005, 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28269 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505791+0000", 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505794+0000", 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505801+0000", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.505806+0000", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507002+0000", 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011957960000000001 2024-09-18T01:08:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507087+0000", 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4950999999999994e-05 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012786899999999999 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511285+0000", 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040708300000000001 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513694+0000", 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024081419999999998 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513721+0000", 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7727999999999999e-05 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513757+0000", 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6155000000000002e-05 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28273 3.a 3:5615eb86:::rbd_data.1176e641238f.00000000000000a0:head [delete] snapc 12=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.527517633000002, 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006596752, 2024-09-18T01:08:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28273 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507210+0000", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507211+0000", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0699999999999996e-07 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507215+0000", 2024-09-18T01:08:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9009999999999996e-06 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507223+0000", 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6119999999999999e-06 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507228+0000", 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507443+0000", 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021441099999999999 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507487+0000", 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4620000000000003e-05 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.507576+0000", 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8218000000000001e-05 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.511330+0000", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037542029999999998 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513762+0000", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024321130000000001 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513785+0000", 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3237000000000001e-05 2024-09-18T01:08:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.513807+0000", 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1509e-05 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:28508 3.10 e52/51 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head v 52'79, mlcod=0'0)", 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.399455441999997, 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0071722770000000003, 2024-09-18T01:08:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635272+0000", 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635273+0000", 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635283+0000", 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8369999999999992e-06 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635293+0000", 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0487e-05 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635302+0000", 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0140000000000004e-06 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635375+0000", 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2527000000000005e-05 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.635397+0000", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2367000000000001e-05 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642399+0000", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070015950000000002 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.642444+0000", 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5834999999999997e-05 2024-09-18T01:08:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.46366555, 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.411674407, 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:08:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.411597610999999, 2024-09-18T01:08:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:08:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.407092457999999, 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:08:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:08:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.406836662, 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:08:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.402811334000001, 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:08:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.402670184, 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:08:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.402437280000001, 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:08:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.399818218, 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:43.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:08:43.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.377479184, 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:08:43.142 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:43.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:08:43.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.147 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:43.148 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:43.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:43.150 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:43.150 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:43.148+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:43.250 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:43.249+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:43.254 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:43.254 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:43.254 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:43.254 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:43.263 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:43.351 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:43.349+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:43.366 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:43.367 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:43.367 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:43.367 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:43.367 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:43.375 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:43.451 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:43.450+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:43.481 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.581397734999996, 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:43.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:43.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:43.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:3016 3.d 3:b44a4505:::rbd_data.11769efc726f.00000000000000b7:head [write 0~360448 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:43.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.188218437000003, 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016196779000000001, 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3016 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290988+0000", 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.290989+0000", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291314+0000", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032540500000000002 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291324+0000", 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7550000000000006e-06 2024-09-18T01:08:43.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.291328+0000", 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2390000000000004e-06 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300773+0000", 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0094451150000000005 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300852+0000", 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8913999999999997e-05 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.300939+0000", 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7196999999999999e-05 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.306767+0000", 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058272970000000004 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307101+0000", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033414099999999998 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307151+0000", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0816999999999998e-05 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.307185+0000", 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.328e-05 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9169 3.d 3:b22c789a:::rbd_data.11762d3aac1.000000000000000b:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~30565 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.821487136999998, 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035450943999999998, 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9169 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657719+0000", 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657720+0000", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4e-07 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674102+0000", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016382088999999999 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674122+0000", 2024-09-18T01:08:43.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9596e-05 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674133+0000", 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1124000000000001e-05 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674207+0000", 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4023000000000006e-05 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674369+0000", 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016215399999999999 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.674661+0000", 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291384 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692828+0000", 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018167822 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693033+0000", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020490699999999999 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693119+0000", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6262999999999997e-05 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.693170+0000", 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742000000000003e-05 2024-09-18T01:08:43.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:14895 3.e 3:76162306:::rbd_data.117650cf82f9.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 55.542099987999997, 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026379288000000001, 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14895 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937106+0000", 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937108+0000", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.28e-06 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937113+0000", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.293e-06 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937126+0000", 2024-09-18T01:08:43.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2597e-05 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937132+0000", 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937180+0000", 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.846e-05 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.937250+0000", 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9239000000000006e-05 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.941566+0000", 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043169380000000002 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963398+0000", 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021831273000000002 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963409+0000", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1721e-05 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963447+0000", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299000000000003e-05 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:47.963486+0000", 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8974999999999999e-05 2024-09-18T01:08:43.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17690 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 360448~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.849778264000001, 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016584738000000002, 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17690 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629428+0000", 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000001e-07 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629430+0000", 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629436+0000", 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4820000000000002e-06 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629439+0000", 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637931+0000", 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084918300000000006 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637999+0000", 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8528999999999994e-05 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646013+0000", 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0080135899999999993 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.512 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17691 3.e 3:76efe84a:::rbd_data.1176492bc9ca.0000000000000003:f [read 458752~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.849763166999999, 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017545595000000001, 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17691 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629443+0000", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629444+0000", 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629445+0000", 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4279999999999999e-06 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629448+0000", 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3179999999999998e-06 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629451+0000", 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.464e-06 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646045+0000", 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016594090999999998 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646149+0000", 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104467 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646989+0000", 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083955399999999995 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17696 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 32768~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.849752017, 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018100321999999999, 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17696 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629454+0000", 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629455+0000", 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629456+0000", 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.215e-06 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629459+0000", 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-18T01:08:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629461+0000", 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5579999999999999e-06 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647025+0000", 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017563921 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647177+0000", 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015128799999999999 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647555+0000", 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037810099999999999 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17697 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:f [read 131072~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.849742380000002, 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019691401000000001, 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17697 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629464+0000", 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:08:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629465+0000", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.093e-06 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629468+0000", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.629471+0000", 2024-09-18T01:08:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3039999999999999e-06 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647596+0000", 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018125596000000001 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647681+0000", 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5198999999999995e-05 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649155+0000", 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014740389999999999 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17711 3.e 3:709fbb95:::rbd_data.1176492bc9ca.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.845830331999998, 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015990583999999999, 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17711 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633376+0000", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633377+0000", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633380+0000", 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8380000000000004e-06 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633391+0000", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0335e-05 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.633398+0000", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649198+0000", 2024-09-18T01:08:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015799732 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649280+0000", 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1829000000000005e-05 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.649367+0000", 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6817000000000006e-05 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:38004 3.15 3:abed255a:::rbd_data.1176f27201d2.0000000000000001:head [write 0~4194304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.782329081999997, 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030253795, 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 38004 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696877+0000", 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.696878+0000", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705325+0000", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084461250000000005 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705338+0000", 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3672999999999999e-05 2024-09-18T01:08:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705345+0000", 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2999999999999998e-06 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705399+0000", 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4954999999999998e-05 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705499+0000", 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9556999999999996e-05 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.705590+0000", 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1458000000000004e-05 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.711484+0000", 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058935719999999997 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727015+0000", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01553087 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727079+0000", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4474999999999995e-05 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:06.727131+0000", 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1743000000000002e-05 2024-09-18T01:08:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43586 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5816320~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084346363999998, 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015218535, 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43586 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394860+0000", 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2499999999999999e-07 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394861+0000", 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394864+0000", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394867+0000", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.409942+0000", 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015075312 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410015+0000", 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3090000000000007e-05 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410079+0000", 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3126999999999994e-05 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43587 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 5980160~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084336497999999, 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015359998, 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43587 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394870+0000", 2024-09-18T01:08:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394871+0000", 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4200000000000004e-07 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394874+0000", 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.959e-06 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394876+0000", 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4399999999999999e-06 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410105+0000", 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015228759 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410155+0000", 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9987000000000001e-05 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410230+0000", 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4697999999999995e-05 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43588 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6144000~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084327133999999, 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015489108999999999, 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43588 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394879+0000", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394880+0000", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394884+0000", 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6829999999999999e-06 2024-09-18T01:08:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394888+0000", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3409999999999999e-06 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394890+0000", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.543e-06 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410258+0000", 2024-09-18T01:08:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015368371 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410303+0000", 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410368+0000", 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5747999999999995e-05 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43589 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6307840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084312566000001, 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015665806000000001, 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43589 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394894+0000", 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394895+0000", 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0079999999999999e-06 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394904+0000", 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.234e-06 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394909+0000", 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410406+0000", 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015497273000000001 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410486+0000", 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9722999999999995e-05 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410560+0000", 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4008e-05 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43590 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6471680~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084293484, 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01587189, 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43590 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394913+0000", 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8500000000000002e-07 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394915+0000", 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3239999999999999e-06 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394918+0000", 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8789999999999996e-06 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394921+0000", 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5620000000000002e-06 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410601+0000", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015679648000000001 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410683+0000", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2826999999999998e-05 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410785+0000", 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101365 2024-09-18T01:08:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43591 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6635520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084282260000002, 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015982698, 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43591 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394924+0000", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394925+0000", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.006e-06 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394928+0000", 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394931+0000", 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5720000000000001e-06 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410803+0000", 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015871962999999999 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410847+0000", 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3912000000000002e-05 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410907+0000", 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0025999999999998e-05 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:43592 3.e 3:75fdc8a6:::rbd_data.1176f27201d2.0000000000000005:19 [read 6799360~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.084257903000001, 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016170249000000001, 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43592 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394949+0000", 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394950+0000", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.161e-06 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394955+0000", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9089999999999999e-06 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.394958+0000", 2024-09-18T01:08:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.593e-06 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.410941+0000", 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015983431999999999 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411019+0000", 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8325999999999995e-05 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:11.411119+0000", 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9463999999999995e-05 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50421 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.246265831999999, 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020565198, 2024-09-18T01:08:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50421 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232941+0000", 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232946+0000", 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3209999999999998e-06 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232956+0000", 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0368e-05 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.232963+0000", 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2400000000000001e-06 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233001+0000", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7298000000000001e-05 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.233052+0000", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1659000000000001e-05 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.236268+0000", 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003215894 2024-09-18T01:08:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253414+0000", 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017146122999999999 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253429+0000", 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4674e-05 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253468+0000", 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.896e-05 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.253506+0000", 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7793999999999998e-05 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50514 3.11 3:8aff915a:::rbd_data.11761711706f.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.08341536, 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015404816999999999, 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50514 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395791+0000", 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395792+0000", 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6300000000000005e-07 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395796+0000", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4490000000000001e-06 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395805+0000", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8580000000000006e-06 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395810+0000", 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395844+0000", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769000000000001e-05 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.395895+0000", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0930000000000002e-05 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.398279+0000", 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023842569999999999 2024-09-18T01:08:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411026+0000", 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012747 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411078+0000", 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2296000000000002e-05 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411143+0000", 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4561999999999996e-05 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.411196+0000", 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3152999999999997e-05 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:50582 3.11 3:8f8a9dba:::rbd_data.11761711706f.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.955438063999999, 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027666276, 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50582 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523768+0000", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523769+0000", 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8599999999999997e-07 2024-09-18T01:08:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523774+0000", 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523783+0000", 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5149999999999995e-06 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523788+0000", 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523838+0000", 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9438999999999999e-05 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.523930+0000", 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1978000000000006e-05 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.528122+0000", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004192749 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551279+0000", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023157015999999999 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551339+0000", 2024-09-18T01:08:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9410000000000002e-05 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551380+0000", 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0927000000000003e-05 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:25.551435+0000", 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4846999999999999e-05 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:43.593 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:43.594 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:43.593+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:43.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:43.695 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:43.694+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:43.700 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:43.700 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:43.700 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:43.700 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:43.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:43.796 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:43.795+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:43.812 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:43.812 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:43.821 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:43.897 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:43.896+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:43.934 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8309 3.1b 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head [write 0~1503721 [fadvise_sequential+fadvise_nocache],zero 1503721~1117719,write 2621440~990465 [fadvise_sequential+fadvise_nocache],zero 3611905~58111,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.345446371999998, 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026761718, 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8309 2024-09-18T01:08:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586563+0000", 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.586564+0000", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0640000000000001e-06 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589632+0000", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030682130000000002 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589658+0000", 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6038999999999999e-05 2024-09-18T01:08:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589669+0000", 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1358e-05 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589749+0000", 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9773999999999999e-05 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.589899+0000", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149522 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590399+0000", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050024900000000003 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610812+0000", 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020412725 2024-09-18T01:08:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613210+0000", 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023980830000000001 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613281+0000", 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1458999999999995e-05 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613324+0000", 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3232000000000002e-05 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8322 3.1b 3:d83a11e2:::rbd_data.1176ec0d676b.0000000000000001:7 [sparse-read 524288~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.341812619000002, 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019995592, 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8322 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590196+0000", 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590197+0000", 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590199+0000", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5179999999999999e-06 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590206+0000", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0230000000000004e-06 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590212+0000", 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:08:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605923+0000", 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01571123 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.605976+0000", 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2405999999999999e-05 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610192+0000", 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004216292 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8338 3.1b 3:d937b864:::rbd_data.1176ec0d676b.000000000000001f:7 [sparse-read 1048576~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.341789050999999, 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020555339999999998, 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8338 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590220+0000", 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590223+0000", 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590229+0000", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.590234+0000", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0180000000000001e-06 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610240+0000", 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.020005832000000001 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610338+0000", 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8463999999999998e-05 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610775+0000", 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043714300000000003 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8347 3.1 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head [write 0~1572864 [fadvise_sequential+fadvise_nocache],zero 1572864~1048576,write 2621440~524288 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 6291456~65536,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache],zero 6553600~65536,write 6619136~32768 [fadvise_sequential+fadvise_nocache],zero 6651904~98304,write 6750208~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.340213648999999, 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021195352000000001, 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8347 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591795+0000", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.591798+0000", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.305e-06 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594507+0000", 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027091939999999998 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594522+0000", 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4745e-05 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594532+0000", 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594609+0000", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6841999999999994e-05 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.594801+0000", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019222600000000001 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.595333+0000", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053156799999999995 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610835+0000", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015502658000000001 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612848+0000", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020127779999999998 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612938+0000", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9906000000000003e-05 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.612991+0000", 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.257e-05 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.335780558000003, 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8392 3.1b 3:d80f88e6:::rbd_data.1176ec0d676b.0000000000000025:7 [sparse-read 262144~262144 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.334458380999997, 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019804817999999998, 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8392 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597551+0000", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597554+0000", 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6450000000000001e-06 2024-09-18T01:08:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597575+0000", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1263e-05 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597580+0000", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.181e-06 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613367+0000", 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.015786337000000001 2024-09-18T01:08:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.613451+0000", 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4383999999999997e-05 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.617355+0000", 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039045669999999998 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8395 3.1f 3:fc759571:::rbd_data.11762d3aac1.0000000000000004:head [write 0~2621440 [fadvise_sequential+fadvise_nocache],zero 2621440~1048576,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.330404569999999, 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023478328999999999, 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8395 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.968 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601604+0000", 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.601607+0000", 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603648+0000", 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002041368 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603666+0000", 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7127999999999999e-05 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.603676+0000", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0015e-05 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606819+0000", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031429790000000002 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.606945+0000", 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126697 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.607771+0000", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082578700000000003 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.619931+0000", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012160261 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.624975+0000", 2024-09-18T01:08:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005043184 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625040+0000", 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5458999999999998e-05 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.625083+0000", 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.282e-05 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8786 3.1 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3174579 [fadvise_sequential+fadvise_nocache],zero 4747443~495437,write 5242880~381 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.304716141, 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026117731000000002, 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8786 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627293+0000", 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.627294+0000", 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9400000000000004e-07 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629631+0000", 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023369010000000002 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629653+0000", 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1997999999999998e-05 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629664+0000", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1418e-05 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629756+0000", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1429999999999997e-05 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.629904+0000", 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000148395 2024-09-18T01:08:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.630210+0000", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306578 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.649555+0000", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019344686 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653292+0000", 2024-09-18T01:08:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037366819999999999 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653359+0000", 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6878000000000006e-05 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.653411+0000", 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1870999999999999e-05 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.274177936000001, 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:08:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:9166 3.18 e34/23 3:1877bf1c:::rbd_data.11762d3aac1.0000000000000009:head v 34'6, mlcod=29'4)", 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.273688243999999, 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025434846000000001, 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.658321+0000", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660932+0000", 2024-09-18T01:08:43.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026106369999999999 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660949+0000", 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6920000000000001e-05 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660957+0000", 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661066+0000", 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010849399999999999 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661097+0000", 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0966999999999997e-05 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683685+0000", 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.022588746999999999 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.683756+0000", 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999994e-05 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 46.209328548000002, 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:08:43.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:08:43.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:43.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41072 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7176192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038290916000001, 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018883337, 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:43.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41072 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:43.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893718+0000", 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893720+0000", 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0329999999999998e-06 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:43.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893726+0000", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893740+0000", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3614999999999999e-05 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912406+0000", 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018665627000000001 2024-09-18T01:08:44.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912494+0000", 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8598999999999997e-05 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912601+0000", 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107236 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41073 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7340032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038262150000001, 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018980520000000001, 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41073 2024-09-18T01:08:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893747+0000", 2024-09-18T01:08:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893749+0000", 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893756+0000", 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893761+0000", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7720000000000002e-06 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912622+0000", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018861822 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912669+0000", 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6161000000000001e-05 2024-09-18T01:08:44.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912727+0000", 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8888000000000003e-05 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41074 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7503872~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038238903, 2024-09-18T01:08:44.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019183375999999999, 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41074 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893770+0000", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893773+0000", 2024-09-18T01:08:44.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893779+0000", 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893784+0000", 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.887e-06 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912756+0000", 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018972671 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912850+0000", 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3843000000000003e-05 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912953+0000", 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010333200000000001 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41075 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7667712~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038218899999997, 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019280973999999999, 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41075 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893790+0000", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.53e-07 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893793+0000", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893798+0000", 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8370000000000002e-06 2024-09-18T01:08:44.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893803+0000", 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6770000000000003e-06 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.912968+0000", 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019165396000000001 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913013+0000", 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913071+0000", 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8143000000000002e-05 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41076 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7831552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:44.015 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038200257, 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019470142999999999, 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41076 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:44.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893809+0000", 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893811+0000", 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893817+0000", 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8100000000000003e-06 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893822+0000", 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913105+0000", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.01928289 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913183+0000", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8003000000000005e-05 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913279+0000", 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6386999999999997e-05 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41077 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 7995392~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038181715, 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019586454, 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41077 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893827+0000", 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893828+0000", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893830+0000", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0949999999999998e-06 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893836+0000", 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7549999999999997e-06 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893842+0000", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913309+0000", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019466831 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913357+0000", 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8395e-05 2024-09-18T01:08:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913414+0000", 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6864000000000003e-05 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41078 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8159232~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038161129000002, 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019771668999999999, 2024-09-18T01:08:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41078 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893848+0000", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893864+0000", 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6024000000000001e-05 2024-09-18T01:08:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893871+0000", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5849999999999996e-06 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893876+0000", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913446+0000", 2024-09-18T01:08:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019570506000000001 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913523+0000", 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6520999999999995e-05 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913620+0000", 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6744999999999996e-05 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41079 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:19 [read 8323072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.038116858999999, 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019915457000000001, 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41079 2024-09-18T01:08:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893892+0000", 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1e-07 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893894+0000", 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.906e-06 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893903+0000", 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0690000000000008e-06 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.893907+0000", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1729999999999999e-06 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913658+0000", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019751510999999999 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913735+0000", 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6452999999999995e-05 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913808+0000", 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3034999999999994e-05 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:41083 3.12 3:4ce07a25:::rbd_data.1176f27201d2.0000000000000002:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.037136666999999, 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019117418000000001, 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 41083 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894872+0000", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894873+0000", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894878+0000", 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894889+0000", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1313e-05 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.894897+0000", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8099999999999998e-06 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913842+0000", 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018944538 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913913+0000", 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1750999999999998e-05 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.913990+0000", 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6445000000000004e-05 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.038 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:44.039 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-18T01:08:44.040 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:44.038+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:44.140 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:44.139+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:44.152 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:44.152 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:44.201 INFO:teuthology.orchestra.run.smithi120.stdout:1862 read 0xb528360 thru 0xb535048 (0xcce9 bytes) 2024-09-18T01:08:44.201 INFO:teuthology.orchestra.run.smithi120.stdout:1863 read 0x42a78d1 thru 0x42acae0 (0x5210 bytes) 2024-09-18T01:08:44.201 INFO:teuthology.orchestra.run.smithi120.stdout:1865 write 0x4a76d52 thru 0x4a79ba2 (0x2e51 bytes) 2024-09-18T01:08:44.223 INFO:teuthology.orchestra.run.smithi120.stdout:1867 trunc from 0xe05a548 to 0x138a3d3 2024-09-18T01:08:44.237 INFO:teuthology.orchestra.run.smithi120.stdout:1868 read 0x10f23da thru 0x10f5914 (0x353b bytes) 2024-09-18T01:08:44.242 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:44.241+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:44.259 INFO:teuthology.orchestra.run.smithi120.stdout:1869 writesame 0xe795b3c thru 0xe79f57d data_size 0x37(0x9a42 bytes) 2024-09-18T01:08:44.273 INFO:teuthology.orchestra.run.smithi120.stdout:1870 trunc from 0xe79f57e to 0x2d36a8f 2024-09-18T01:08:44.276 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.276 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:44.277 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:44.277 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:44.277 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:44.293 INFO:teuthology.orchestra.run.smithi120.stdout:1872 writesame 0x9860146 thru 0x9869f9b data_size 0xd(0x9e56 bytes) 2024-09-18T01:08:44.293 INFO:teuthology.orchestra.run.smithi120.stdout:1873 read 0x12476b6 thru 0x12511bf (0x9b0a bytes) 2024-09-18T01:08:44.293 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.294 INFO:teuthology.orchestra.run.smithi120.stdout:1875 writesame 0x5c416b thru 0x5c425d data_size 0x1(0xf3 bytes) 2024-09-18T01:08:44.298 INFO:teuthology.orchestra.run.smithi120.stdout:1876 trunc from 0x9869f9c to 0xc4a0d7f 2024-09-18T01:08:44.304 INFO:teuthology.orchestra.run.smithi120.stdout:1878 writesame 0x34351f4 thru 0x343cdb0 data_size 0x1(0x7bbd bytes) 2024-09-18T01:08:44.330 INFO:teuthology.orchestra.run.smithi120.stdout:1879 trunc from 0xc4a0d7f to 0xe5c26d 2024-09-18T01:08:44.339 INFO:teuthology.orchestra.run.smithi120.stdout:1881 punch from 0x30027e to 0x307065, (0x6de7 bytes) 2024-09-18T01:08:44.339 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.339 INFO:teuthology.orchestra.run.smithi120.stdout:1884 read 0x164908 thru 0x16a295 (0x598e bytes) 2024-09-18T01:08:44.342 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:44.342+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:44.348 INFO:teuthology.orchestra.run.smithi120.stdout:1886 writesame 0x3d00117 thru 0x3d0720d data_size 0xb(0x70f7 bytes) 2024-09-18T01:08:44.361 INFO:teuthology.orchestra.run.smithi120.stdout:1887 write 0x799d1b9 thru 0x79a78d8 (0xa720 bytes) 2024-09-18T01:08:44.364 INFO:teuthology.orchestra.run.smithi120.stdout:1888 trunc from 0x79a78d9 to 0x6ce8a34 2024-09-18T01:08:44.381 INFO:teuthology.orchestra.run.smithi120.stdout:1889 trunc from 0x6ce8a34 to 0x4cb594 2024-09-18T01:08:44.390 INFO:teuthology.orchestra.run.smithi120.stdout:1891 punch from 0x388745 to 0x393ed8, (0xb793 bytes) 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228646378999997, 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:44.409 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:44.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228629811000005, 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.414 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:44.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.418 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228613624999994, 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:44.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:44.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228602056, 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.423 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:44.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228592689999999, 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.428 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:44.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228580507000004, 2024-09-18T01:08:44.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.433 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:44.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228564812000002, 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:44.437 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228547657999997, 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228525492000003, 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228509041999999, 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228499588000005, 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228465537000005, 2024-09-18T01:08:44.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:44.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228445812999993, 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:44.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.469 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228427576000001, 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:44.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228407000999994, 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:44.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228387773999998, 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:44.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228368420999999, 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:44.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:44.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:44.487 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.228349765999994, 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.488 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:44.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.219567816999998, 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:44.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:44.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:44.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.500 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:50519 3.19 3:9a8c2274:::rbd_data.11761711706f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.988116708, 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026218866, 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 50519 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418749+0000", 2024-09-18T01:08:44.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.85e-07 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418754+0000", 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2910000000000001e-06 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418764+0000", 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418769+0000", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1789999999999999e-06 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418814+0000", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4350000000000001e-05 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418863+0000", 2024-09-18T01:08:44.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9264000000000001e-05 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.423079+0000", 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042156269999999996 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444883+0000", 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021804975000000001 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444896+0000", 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2344000000000001e-05 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444929+0000", 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.269e-05 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444967+0000", 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8818e-05 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.507 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.508 INFO:teuthology.orchestra.run.smithi120.stdout:1892 write 0xabc409f thru 0xabcf557 (0xb4b9 bytes) 2024-09-18T01:08:44.508 INFO:teuthology.orchestra.run.smithi120.stdout:1893 writesame 0x6d06efd thru 0x6d0c36d data_size 0x1(0x5471 bytes) 2024-09-18T01:08:44.509 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:44.511 INFO:teuthology.orchestra.run.smithi120.stdout:1895 read 0x6d86a2d thru 0x6d92ddf (0xc3b3 bytes) 2024-09-18T01:08:44.511 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.511 INFO:teuthology.orchestra.run.smithi120.stdout:1898 write 0x6f8b9c6 thru 0x6f9b2ea (0xf925 bytes) 2024-09-18T01:08:44.511 INFO:teuthology.orchestra.run.smithi120.stdout:1899 write 0x88c62f thru 0x890b14 (0x44e6 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1900 trunc from 0xabcf558 to 0x7ec1ff7 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1902 read 0x61f648a thru 0x62053d5 (0xef4c bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1905 punch from 0xfe0f60 to 0xfef34e, (0xe3ee bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1906 read 0x2b33ff9 thru 0x2b378d1 (0x38d9 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1908 read 0x685b3fa thru 0x685f2df (0x3ee6 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1909 write 0x2cba5c0 thru 0x2cc1659 (0x709a bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1910 punch from 0x28f4f10 to 0x28febb1, (0x9ca1 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1911 write 0xce05fe2 thru 0xce115fe (0xb61d bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1912 writesame 0x7c884e3 thru 0x7c910b7 data_size 0x1(0x8bd5 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1914 trunc from 0xce115ff to 0x4bd2d5f 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1915 punch from 0xcad65b to 0xcb69ce, (0x9373 bytes) 2024-09-18T01:08:44.512 INFO:teuthology.orchestra.run.smithi120.stdout:1916 write 0xc4d8f99 thru 0xc4dcde1 (0x3e49 bytes) 2024-09-18T01:08:44.513 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:44.510+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:44.516 INFO:teuthology.orchestra.run.smithi120.stdout:1919 write 0x64c1c7d thru 0x64c3b3e (0x1ec2 bytes) 2024-09-18T01:08:44.519 INFO:teuthology.orchestra.run.smithi120.stdout:1920 writesame 0x8895780 thru 0x889bfd4 data_size 0x1d(0x6855 bytes) 2024-09-18T01:08:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:1922 read 0x78a8e5a thru 0x78ac4cd (0x3674 bytes) 2024-09-18T01:08:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:1923 read 0xa4d109c thru 0xa4d494f (0x38b4 bytes) 2024-09-18T01:08:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:1924 read 0x6b91c4c thru 0x6b99992 (0x7d47 bytes) 2024-09-18T01:08:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:1927 read 0x378fdf8 thru 0x37902d9 (0x4e2 bytes) 2024-09-18T01:08:44.525 INFO:teuthology.orchestra.run.smithi120.stdout:1928 read 0x3cabdea thru 0x3cb73f1 (0xb608 bytes) 2024-09-18T01:08:44.525 INFO:teuthology.orchestra.run.smithi120.stdout:1929 punch from 0x10e7936 to 0x10e9682, (0x1d4c bytes) 2024-09-18T01:08:44.525 INFO:teuthology.orchestra.run.smithi120.stdout:1930 read 0x7c788e0 thru 0x7c7dc7b (0x539c bytes) 2024-09-18T01:08:44.530 INFO:teuthology.orchestra.run.smithi120.stdout:1931 write 0xc5a1aa5 thru 0xc5a7923 (0x5e7f bytes) 2024-09-18T01:08:44.536 INFO:teuthology.orchestra.run.smithi120.stdout:1933 trunc from 0xc5a7924 to 0x878ec28 2024-09-18T01:08:44.547 INFO:teuthology.orchestra.run.smithi120.stdout:1934 read 0x5e54665 thru 0x5e54ea1 (0x83d bytes) 2024-09-18T01:08:44.547 INFO:teuthology.orchestra.run.smithi120.stdout:1936 punch from 0x37f24b1 to 0x37faf4d, (0x8a9c bytes) 2024-09-18T01:08:44.568 INFO:teuthology.orchestra.run.smithi120.stdout:1937 write 0xe60a441 thru 0xe6190b9 (0xec79 bytes) 2024-09-18T01:08:44.572 INFO:teuthology.orchestra.run.smithi120.stdout:1938 write 0x9a9d189 thru 0x9aac503 (0xf37b bytes) 2024-09-18T01:08:44.576 INFO:teuthology.orchestra.run.smithi120.stdout:1940 read 0xcec5ea1 thru 0xcec6a48 (0xba8 bytes) 2024-09-18T01:08:44.586 INFO:teuthology.orchestra.run.smithi120.stdout:1941 trunc from 0xe6190ba to 0x5b60b9d 2024-09-18T01:08:44.605 INFO:teuthology.orchestra.run.smithi120.stdout:1942 trunc from 0x5b60b9d to 0x96a5e7f 2024-09-18T01:08:44.610 INFO:teuthology.orchestra.run.smithi120.stdout:1943 punch from 0x62c31f0 to 0x62c93bf, (0x61cf bytes) 2024-09-18T01:08:44.610 INFO:teuthology.orchestra.run.smithi120.stdout:1944 punch from 0x7d8389d to 0x7d87ce2, (0x4445 bytes) 2024-09-18T01:08:44.611 INFO:teuthology.orchestra.run.smithi120.stdout:1945 write 0x295c15a thru 0x2965c64 (0x9b0b bytes) 2024-09-18T01:08:44.611 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:44.610+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:44.612 INFO:teuthology.orchestra.run.smithi120.stdout:1947 writesame 0x83ecabd thru 0x83ef2a5 data_size 0x1(0x27e9 bytes) 2024-09-18T01:08:44.618 INFO:teuthology.orchestra.run.smithi120.stdout:1948 writesame 0x4414ce5 thru 0x44171eb data_size 0x1(0x2507 bytes) 2024-09-18T01:08:44.627 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:44.628 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.632 INFO:teuthology.orchestra.run.smithi120.stdout:1949 write 0xb066e9f thru 0xb06904d (0x21af bytes) 2024-09-18T01:08:44.634 INFO:teuthology.orchestra.run.smithi120.stdout:1951 read 0x6a21957 thru 0x6a2772e (0x5dd8 bytes) 2024-09-18T01:08:44.634 INFO:teuthology.orchestra.run.smithi120.stdout:1952 read 0x9a53f95 thru 0x9a61850 (0xd8bc bytes) 2024-09-18T01:08:44.634 INFO:teuthology.orchestra.run.smithi120.stdout:1954 read 0x900f817 thru 0x9015448 (0x5c32 bytes) 2024-09-18T01:08:44.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:44.642 INFO:teuthology.orchestra.run.smithi120.stdout:1955 write 0xcfaa9e2 thru 0xcfb83b0 (0xd9cf bytes) 2024-09-18T01:08:44.652 INFO:teuthology.orchestra.run.smithi120.stdout:1956 writesame 0xe4d2e11 thru 0xe4e10bb data_size 0x1(0xe2ab bytes) 2024-09-18T01:08:44.652 INFO:teuthology.orchestra.run.smithi120.stdout:1957 writesame 0x6552faa thru 0x655a812 data_size 0x4b(0x7869 bytes) 2024-09-18T01:08:44.657 INFO:teuthology.orchestra.run.smithi120.stdout:1958 writesame 0xd0b56e7 thru 0xd0bf1e4 data_size 0x22(0x9afe bytes) 2024-09-18T01:08:44.660 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.660 INFO:teuthology.orchestra.run.smithi120.stdout:1962 read 0x179b994 thru 0x17a625e (0xa8cb bytes) 2024-09-18T01:08:44.661 INFO:teuthology.orchestra.run.smithi120.stdout:1966 trunc from 0xe4e10bc to 0xe706ec6 2024-09-18T01:08:44.683 INFO:teuthology.orchestra.run.smithi120.stdout:1968 trunc from 0xe706ec6 to 0x11161fb 2024-09-18T01:08:44.707 INFO:teuthology.orchestra.run.smithi120.stdout:1969 trunc from 0x11161fb to 0x6848033 2024-09-18T01:08:44.713 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:44.712+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:44.726 INFO:teuthology.orchestra.run.smithi120.stdout:1970 writesame 0xb3f6ac6 thru 0xb3f6ed5 data_size 0x1(0x410 bytes) 2024-09-18T01:08:44.726 INFO:teuthology.orchestra.run.smithi120.stdout:1971 writesame 0x1d952c4 thru 0x1d96164 data_size 0x7(0xea1 bytes) 2024-09-18T01:08:44.728 INFO:teuthology.orchestra.run.smithi120.stdout:1972 read 0x706092b thru 0x706f2ce (0xe9a4 bytes) 2024-09-18T01:08:44.728 INFO:teuthology.orchestra.run.smithi120.stdout:1973 read 0x984a7df thru 0x9859fc6 (0xf7e8 bytes) 2024-09-18T01:08:44.728 INFO:teuthology.orchestra.run.smithi120.stdout:1974 punch from 0x6942aae to 0x6947daa, (0x52fc bytes) 2024-09-18T01:08:44.728 INFO:teuthology.orchestra.run.smithi120.stdout:1976 write 0x212ef51 thru 0x213d40c (0xe4bc bytes) 2024-09-18T01:08:44.731 INFO:teuthology.orchestra.run.smithi120.stdout:1977 write 0x45fdb60 thru 0x45ffaef (0x1f90 bytes) 2024-09-18T01:08:44.746 INFO:teuthology.orchestra.run.smithi120.stdout:1978 trunc from 0xb3f6ed6 to 0xd325b 2024-09-18T01:08:44.761 INFO:teuthology.orchestra.run.smithi120.stdout:1979 trunc from 0xd325b to 0x4109c52 2024-09-18T01:08:44.761 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:44.762 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:44.778 INFO:teuthology.orchestra.run.smithi120.stdout:1980 write 0x830caa7 thru 0x830e444 (0x199e bytes) 2024-09-18T01:08:44.781 INFO:teuthology.orchestra.run.smithi120.stdout:1981 read 0x5289e7d thru 0x529291d (0x8aa1 bytes) 2024-09-18T01:08:44.786 INFO:teuthology.orchestra.run.smithi120.stdout:1982 write 0x8a34b2b thru 0x8a3cb53 (0x8029 bytes) 2024-09-18T01:08:44.791 INFO:teuthology.orchestra.run.smithi120.stdout:1983 trunc from 0x8a3cb54 to 0xb2f11ee 2024-09-18T01:08:44.796 INFO:teuthology.orchestra.run.smithi120.stdout:1984 write 0xa86a5e9 thru 0xa878aa4 (0xe4bc bytes) 2024-09-18T01:08:44.801 INFO:teuthology.orchestra.run.smithi120.stdout:1986 trunc from 0xb2f11ee to 0xc856511 2024-09-18T01:08:44.806 INFO:teuthology.orchestra.run.smithi120.stdout:1988 punch from 0x6c9608a to 0x6ca3148, (0xd0be bytes) 2024-09-18T01:08:44.814 INFO:teuthology.orchestra.run.smithi120.stdout:1989 writesame 0xeb0f764 thru 0xeb0fba5 data_size 0x5(0x442 bytes) 2024-09-18T01:08:44.814 INFO:teuthology.orchestra.run.smithi120.stdout:1990 writesame 0xe030bb8 thru 0xe03692f data_size 0xc(0x5d78 bytes) 2024-09-18T01:08:44.814 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:44.813+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:44.823 INFO:teuthology.orchestra.run.smithi120.stdout:1991 write 0xee55058 thru 0xee5b7c9 (0x6772 bytes) 2024-09-18T01:08:44.825 INFO:teuthology.orchestra.run.smithi120.stdout:1992 read 0x3948033 thru 0x39539b3 (0xb981 bytes) 2024-09-18T01:08:44.825 INFO:teuthology.orchestra.run.smithi120.stdout:1993 write 0x68c6939 thru 0x68c926c (0x2934 bytes) 2024-09-18T01:08:44.829 INFO:teuthology.orchestra.run.smithi120.stdout:1994 read 0x1209583 thru 0x120be21 (0x289f bytes) 2024-09-18T01:08:44.829 INFO:teuthology.orchestra.run.smithi120.stdout:1995 writesame 0x2941a7f thru 0x294615d data_size 0x1(0x46df bytes) 2024-09-18T01:08:44.845 INFO:teuthology.orchestra.run.smithi120.stdout:1996 trunc from 0xee5b7ca to 0xa4a46b9 2024-09-18T01:08:44.861 INFO:teuthology.orchestra.run.smithi120.stdout:1997 writesame 0x14b942d thru 0x14bb8a1 data_size 0x11(0x2475 bytes) 2024-09-18T01:08:44.864 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:1998 punch from 0x65b9dd3 to 0x65c159e, (0x77cb bytes) 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:1999 punch from 0x7e2aed6 to 0x7e3a6aa, (0xf7d4 bytes) 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:2001 read 0x819f698 thru 0x81a4298 (0x4c01 bytes) 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:2002 punch from 0x9d2586 to 0x9d8b9c, (0x6616 bytes) 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.866 INFO:teuthology.orchestra.run.smithi120.stdout:2004 read 0x23e6f4c thru 0x23e88f7 (0x19ac bytes) 2024-09-18T01:08:44.867 INFO:teuthology.orchestra.run.smithi120.stdout:2005 punch from 0x84bc047 to 0x84c917a, (0xd133 bytes) 2024-09-18T01:08:44.867 INFO:teuthology.orchestra.run.smithi120.stdout:2008 punch from 0x8ed2cb2 to 0x8ed7a14, (0x4d62 bytes) 2024-09-18T01:08:44.867 INFO:teuthology.orchestra.run.smithi120.stdout:2009 write 0xa08e9ea thru 0xa09a0e2 (0xb6f9 bytes) 2024-09-18T01:08:44.870 INFO:teuthology.orchestra.run.smithi120.stdout:2010 read 0x7c1809 thru 0x7c76f2 (0x5eea bytes) 2024-09-18T01:08:44.871 INFO:teuthology.orchestra.run.smithi120.stdout:2011 read 0x16bc8ca thru 0x16c61df (0x9916 bytes) 2024-09-18T01:08:44.871 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.871 INFO:teuthology.orchestra.run.smithi120.stdout:2013 writesame 0x6cf00ce thru 0x6cfece6 data_size 0x1(0xec19 bytes) 2024-09-18T01:08:44.892 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 85.951003186999998, 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.823075146999997, 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.772701087000002, 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.682807870999994, 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.681212122999995, 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.680401504000002, 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.297028693000001, 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.733428635999999, 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.715667549999999, 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.713010847, 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.130363303999999, 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.716049845000001, 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.214350822999997, 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.212773935999998, 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.183026345999998, 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.137383184999997, 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.027646859999997, 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.023993429999997, 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:44.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:44.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:52190 3.4 e82/23 3:2587a2b7:::rbd_data.11762afc950e.0000000000000022:head v 82'135, mlcod=77'133)", 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.28777724, 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038334850000000002, 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:44.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602975+0000", 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3629999999999999e-05 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602984+0000", 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0820000000000005e-06 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602992+0000", 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6920000000000002e-06 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603100+0000", 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107977 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603124+0000", 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4544000000000001e-05 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606735+0000", 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036103580000000001 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606784+0000", 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9697000000000002e-05 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3083267550000004, 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:44.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:44.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:08:44.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:44.991 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:44.992 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:44.991+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:44.993 INFO:teuthology.orchestra.run.smithi120.stdout:2014 read 0x6a6754d thru 0x6a6bf73 (0x4a27 bytes) 2024-09-18T01:08:44.993 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:44.996 INFO:teuthology.orchestra.run.smithi120.stdout:2016 trunc from 0xa4a46b9 to 0x7529506 2024-09-18T01:08:44.996 INFO:teuthology.orchestra.run.smithi120.stdout:2017 read 0x6a9b5b5 thru 0x6aa0d21 (0x576d bytes) 2024-09-18T01:08:44.996 INFO:teuthology.orchestra.run.smithi120.stdout:2018 writesame 0x179c470 thru 0x17a0f83 data_size 0x3e(0x4b14 bytes) 2024-09-18T01:08:44.996 INFO:teuthology.orchestra.run.smithi120.stdout:2019 write 0x7cabfdc thru 0x7cb915e (0xd183 bytes) 2024-09-18T01:08:44.996 INFO:teuthology.orchestra.run.smithi120.stdout:2020 punch from 0x31de5b7 to 0x31e91f2, (0xac3b bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2021 punch from 0x77d2fee to 0x77dcb5f, (0x9b71 bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2022 trunc from 0x7cb915f to 0x3de0500 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2024 write 0x20a99b6 thru 0x20ad5b2 (0x3bfd bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2026 punch from 0x322418c to 0x322efb3, (0xae27 bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2027 punch from 0x1cf0f86 to 0x1cff975, (0xe9ef bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2028 trunc from 0x3de0500 to 0xd078783 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2029 write 0xe40d3f8 thru 0xe4103db (0x2fe4 bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2031 read 0x6a97c39 thru 0x6a98c04 (0xfcc bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2032 punch from 0x1a38f01 to 0x1a4516f, (0xc26e bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:2034 read 0xa317cb5 thru 0xa3270b8 (0xf404 bytes) 2024-09-18T01:08:44.997 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:2042 write 0x2f42765 thru 0x2f44cf1 (0x258d bytes) 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:2043 write 0x603af4f thru 0x603d8ee (0x29a0 bytes) 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:2044 write 0xbf9bc20 thru 0xbfa9857 (0xdc38 bytes) 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:2045 write 0xb309770 thru 0xb30baf6 (0x2387 bytes) 2024-09-18T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout:2046 writesame 0x1b8437f thru 0x1b8e524 data_size 0xc6(0xa1a6 bytes) 2024-09-18T01:08:44.999 INFO:teuthology.orchestra.run.smithi120.stdout:2048 trunc from 0xe4103dc to 0x9aa2a6b 2024-09-18T01:08:45.011 INFO:teuthology.orchestra.run.smithi120.stdout:2050 writesame 0x9358b56 thru 0x935c45b data_size 0x9(0x3906 bytes) 2024-09-18T01:08:45.013 INFO:teuthology.orchestra.run.smithi120.stdout:2051 writesame 0xeedd73 thru 0xef7c68 data_size 0x1(0x9ef6 bytes) 2024-09-18T01:08:45.045 INFO:teuthology.orchestra.run.smithi120.stdout:2053 write 0xdcf74a7 thru 0xdcf973e (0x2298 bytes) 2024-09-18T01:08:45.047 INFO:teuthology.orchestra.run.smithi120.stdout:2054 read 0x61db644 thru 0x61dfe9b (0x4858 bytes) 2024-09-18T01:08:45.061 INFO:teuthology.orchestra.run.smithi120.stdout:2057 trunc from 0xdcf973f to 0x270ab50 2024-09-18T01:08:45.076 INFO:teuthology.orchestra.run.smithi120.stdout:2059 trunc from 0x270ab50 to 0x27da7be 2024-09-18T01:08:45.084 INFO:teuthology.orchestra.run.smithi120.stdout:2060 trunc from 0x27da7be to 0x4172884 2024-09-18T01:08:45.090 INFO:teuthology.orchestra.run.smithi120.stdout:2062 punch from 0x5ad475 to 0x5b088a, (0x3415 bytes) 2024-09-18T01:08:45.090 INFO:teuthology.orchestra.run.smithi120.stdout:2063 read 0x35f1f14 thru 0x35fad3b (0x8e28 bytes) 2024-09-18T01:08:45.093 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:45.092+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:45.106 INFO:teuthology.orchestra.run.smithi120.stdout:2065 write 0xbb28954 thru 0xbb2d32e (0x49db bytes) 2024-09-18T01:08:45.109 INFO:teuthology.orchestra.run.smithi120.stdout:2066 write 0x48df3f4 thru 0x48e1120 (0x1d2d bytes) 2024-09-18T01:08:45.110 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.110 INFO:teuthology.orchestra.run.smithi120.stdout:2068 write 0x782089d thru 0x782f003 (0xe767 bytes) 2024-09-18T01:08:45.113 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.114 INFO:teuthology.orchestra.run.smithi120.stdout:2070 read 0x50e9167 thru 0x50f7bf8 (0xea92 bytes) 2024-09-18T01:08:45.116 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:45.116 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:45.116 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:45.116 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:2071 trunc from 0xbb2d32f to 0x2cff81e 2024-09-18T01:08:45.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:45.144 INFO:teuthology.orchestra.run.smithi120.stdout:2072 write 0x5664d59 thru 0x5665512 (0x7ba bytes) 2024-09-18T01:08:45.146 INFO:teuthology.orchestra.run.smithi120.stdout:2073 punch from 0x1a777d5 to 0x1a81f06, (0xa731 bytes) 2024-09-18T01:08:45.146 INFO:teuthology.orchestra.run.smithi120.stdout:2074 read 0x35ae466 thru 0x35b34b5 (0x5050 bytes) 2024-09-18T01:08:45.147 INFO:teuthology.orchestra.run.smithi120.stdout:2075 write 0x1c71bdf thru 0x1c7dc99 (0xc0bb bytes) 2024-09-18T01:08:45.147 INFO:teuthology.orchestra.run.smithi120.stdout:2076 read 0x3c7549f thru 0x3c769f7 (0x1559 bytes) 2024-09-18T01:08:45.147 INFO:teuthology.orchestra.run.smithi120.stdout:2077 read 0x1104df7 thru 0x1111bb5 (0xcdbf bytes) 2024-09-18T01:08:45.148 INFO:teuthology.orchestra.run.smithi120.stdout:2078 read 0x3d0bf5e thru 0x3d17895 (0xb938 bytes) 2024-09-18T01:08:45.150 INFO:teuthology.orchestra.run.smithi120.stdout:2079 trunc from 0x5665513 to 0x340b416 2024-09-18T01:08:45.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:2082 punch from 0x28ebe9b to 0x28f6035, (0xa19a bytes) 2024-09-18T01:08:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:2083 punch from 0x2534af2 to 0x253a413, (0x5921 bytes) 2024-09-18T01:08:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:2084 read 0xb0888a thru 0xb144f5 (0xbc6c bytes) 2024-09-18T01:08:45.171 INFO:teuthology.orchestra.run.smithi120.stdout:2085 write 0x8820441 thru 0x882e693 (0xe253 bytes) 2024-09-18T01:08:45.179 INFO:teuthology.orchestra.run.smithi120.stdout:2086 trunc from 0x882e694 to 0x58f8c1f 2024-09-18T01:08:45.192 INFO:teuthology.orchestra.run.smithi120.stdout:2087 read 0x4f534b8 thru 0x4f61de4 (0xe92d bytes) 2024-09-18T01:08:45.192 INFO:teuthology.orchestra.run.smithi120.stdout:2088 read 0x5783982 thru 0x578cbf1 (0x9270 bytes) 2024-09-18T01:08:45.193 INFO:teuthology.orchestra.run.smithi120.stdout:2090 punch from 0x42bdda3 to 0x42c4694, (0x68f1 bytes) 2024-09-18T01:08:45.194 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:45.193+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:45.209 INFO:teuthology.orchestra.run.smithi120.stdout:2091 write 0xc26a21e thru 0xc26fe4d (0x5c30 bytes) 2024-09-18T01:08:45.214 INFO:teuthology.orchestra.run.smithi120.stdout:2093 write 0xb882fd7 thru 0xb88d315 (0xa33f bytes) 2024-09-18T01:08:45.218 INFO:teuthology.orchestra.run.smithi120.stdout:2095 write 0x88cfea9 thru 0x88dc4a8 (0xc600 bytes) 2024-09-18T01:08:45.219 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.219 INFO:teuthology.orchestra.run.smithi120.stdout:2097 write 0xac06e26 thru 0xac15539 (0xe714 bytes) 2024-09-18T01:08:45.223 INFO:teuthology.orchestra.run.smithi120.stdout:2098 read 0xb04633c thru 0xb0556fe (0xf3c3 bytes) 2024-09-18T01:08:45.223 INFO:teuthology.orchestra.run.smithi120.stdout:2099 read 0x41db66 thru 0x425c64 (0x80ff bytes) 2024-09-18T01:08:45.224 INFO:teuthology.orchestra.run.smithi120.stdout:2100 write 0xa92a72 thru 0xaa1493 (0xea22 bytes) 2024-09-18T01:08:45.231 INFO:teuthology.orchestra.run.smithi120.stdout:2103 trunc from 0xc26fe4e to 0x800025b 2024-09-18T01:08:45.242 INFO:teuthology.orchestra.run.smithi120.stdout:2104 write 0x2343ccc thru 0x2344a63 (0xd98 bytes) 2024-09-18T01:08:45.244 INFO:teuthology.orchestra.run.smithi120.stdout:2105 write 0x6e778a4 thru 0x6e7f22e (0x798b bytes) 2024-09-18T01:08:45.247 INFO:teuthology.orchestra.run.smithi120.stdout:2106 write 0x972d25 thru 0x97b94d (0x8c29 bytes) 2024-09-18T01:08:45.248 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.255 INFO:teuthology.orchestra.run.smithi120.stdout:2108 write 0x96e882a thru 0x96f81aa (0xf981 bytes) 2024-09-18T01:08:45.259 INFO:teuthology.orchestra.run.smithi120.stdout:2109 write 0x2a0e011 thru 0x2a0e85c (0x84c bytes) 2024-09-18T01:08:45.261 INFO:teuthology.orchestra.run.smithi120.stdout:2111 writesame 0x87acc7a thru 0x87b6c44 data_size 0x1(0x9fcb bytes) 2024-09-18T01:08:45.274 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:45.274 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:45.274 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:45.274 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:45.274 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:45.281 INFO:teuthology.orchestra.run.smithi120.stdout:2113 writesame 0x779ef90 thru 0x77abfdc data_size 0x1b(0xd04d bytes) 2024-09-18T01:08:45.284 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:45.284 INFO:teuthology.orchestra.run.smithi120.stdout:2114 read 0x5e36ce4 thru 0x5e408da (0x9bf7 bytes) 2024-09-18T01:08:45.289 INFO:teuthology.orchestra.run.smithi120.stdout:2115 trunc from 0x96f81ab to 0xd66d7e0 2024-09-18T01:08:45.289 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:45.289 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:45.294 INFO:teuthology.orchestra.run.smithi120.stdout:2116 write 0x5fe01ff thru 0x5fe2b9b (0x299d bytes) 2024-09-18T01:08:45.294 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:45.293+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:45.295 INFO:teuthology.orchestra.run.smithi120.stdout:2117 writesame 0xaa4b894 thru 0xaa52f63 data_size 0x8(0x76d0 bytes) 2024-09-18T01:08:45.297 INFO:teuthology.orchestra.run.smithi120.stdout:2118 punch from 0x1d6c2e6 to 0x1d6ff41, (0x3c5b bytes) 2024-09-18T01:08:45.300 INFO:teuthology.orchestra.run.smithi120.stdout:2119 trunc from 0xd66d7e0 to 0xb26da22 2024-09-18T01:08:45.307 INFO:teuthology.orchestra.run.smithi120.stdout:2120 read 0x855e78f thru 0x856977e (0xaff0 bytes) 2024-09-18T01:08:45.310 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:45.310 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:45.314 INFO:teuthology.orchestra.run.smithi120.stdout:2123 trunc from 0xb26da22 to 0x58f4af1 2024-09-18T01:08:45.330 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:45.331 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:45.348 INFO:teuthology.orchestra.run.smithi120.stdout:2125 write 0xe490f0d thru 0xe4a0acb (0xfbbf bytes) 2024-09-18T01:08:45.352 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.352 INFO:teuthology.orchestra.run.smithi120.stdout:2127 read 0x6ef706e thru 0x6f05263 (0xe1f6 bytes) 2024-09-18T01:08:45.352 INFO:teuthology.orchestra.run.smithi120.stdout:2129 punch from 0x14ebe0d to 0x14ef08f, (0x3282 bytes) 2024-09-18T01:08:45.352 INFO:teuthology.orchestra.run.smithi120.stdout:2131 write 0x369aabb thru 0x36a040e (0x5954 bytes) 2024-09-18T01:08:45.353 INFO:teuthology.orchestra.run.smithi120.stdout:2132 punch from 0x6863c51 to 0x686efeb, (0xb39a bytes) 2024-09-18T01:08:45.353 INFO:teuthology.orchestra.run.smithi120.stdout:2133 write 0xc6b049c thru 0xc6be597 (0xe0fc bytes) 2024-09-18T01:08:45.355 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:45.358 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.358 INFO:teuthology.orchestra.run.smithi120.stdout:2135 punch from 0xdce2d18 to 0xdcf0a9f, (0xdd87 bytes) 2024-09-18T01:08:45.373 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:45.374 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:45.374 INFO:teuthology.orchestra.run.smithi120.stdout:2136 trunc from 0xe4a0acc to 0xd3dff5 2024-09-18T01:08:45.394 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:45.394 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:45.395 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:45.394+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:45.402 INFO:teuthology.orchestra.run.smithi120.stdout:2138 punch from 0x14353b to 0x1441e1, (0xca6 bytes) 2024-09-18T01:08:45.405 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.405 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.406 INFO:teuthology.orchestra.run.smithi120.stdout:2139 trunc from 0xd3dff5 to 0x37d570c 2024-09-18T01:08:45.411 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.411 INFO:teuthology.orchestra.run.smithi120.stdout:2143 write 0x197ab7c thru 0x1988c08 (0xe08d bytes) 2024-09-18T01:08:45.413 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:45.415 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:45.416 INFO:teuthology.orchestra.run.smithi120.stdout:2144 write 0xbd8688 thru 0xbddaf8 (0x5471 bytes) 2024-09-18T01:08:45.432 INFO:teuthology.orchestra.run.smithi120.stdout:2145 writesame 0xb533c3f thru 0xb53c842 data_size 0x74(0x8c04 bytes) 2024-09-18T01:08:45.432 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.432 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:45.432 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:45.433 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-18T01:08:45.433 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:08:45.433 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:08:45.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-18T01:08:45.441 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:45.441 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:45.443 INFO:teuthology.orchestra.run.smithi120.stdout:2147 trunc from 0xb53c843 to 0x2a4a949 2024-09-18T01:08:45.460 INFO:teuthology.orchestra.run.smithi120.stdout:2148 write 0x76c431c thru 0x76cc8ae (0x8593 bytes) 2024-09-18T01:08:45.477 INFO:teuthology.orchestra.run.smithi120.stdout:2149 writesame 0xd9f9756 thru 0xda062f3 data_size 0x43(0xcb9e bytes) 2024-09-18T01:08:45.477 INFO:teuthology.orchestra.run.smithi120.stdout:2151 writesame 0x5aab9a6 thru 0x5ab5bc1 data_size 0xa(0xa21c bytes) 2024-09-18T01:08:45.481 INFO:teuthology.orchestra.run.smithi120.stdout:2152 write 0x99e5537 thru 0x99e8172 (0x2c3c bytes) 2024-09-18T01:08:45.484 INFO:teuthology.orchestra.run.smithi120.stdout:2153 read 0xb3d51ed thru 0xb3d7618 (0x242c bytes) 2024-09-18T01:08:45.485 INFO:teuthology.orchestra.run.smithi120.stdout:2155 write 0x693bfde thru 0x6941d73 (0x5d96 bytes) 2024-09-18T01:08:45.486 INFO:teuthology.orchestra.run.smithi120.stdout:2157 read 0x2c9b355 thru 0x2cab0d5 (0xfd81 bytes) 2024-09-18T01:08:45.486 INFO:teuthology.orchestra.run.smithi120.stdout:2158 read 0xb62b0a thru 0xb67f6a (0x5461 bytes) 2024-09-18T01:08:45.486 INFO:teuthology.orchestra.run.smithi120.stdout:2159 read 0x7990f15 thru 0x799d5a4 (0xc690 bytes) 2024-09-18T01:08:45.496 INFO:teuthology.orchestra.run.smithi120.stdout:2161 write 0xe3ed794 thru 0xe3f9196 (0xba03 bytes) 2024-09-18T01:08:45.496 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:45.495+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:45.499 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.499 INFO:teuthology.orchestra.run.smithi120.stdout:2164 read 0x87e9e6f thru 0x87f8601 (0xe793 bytes) 2024-09-18T01:08:45.508 INFO:teuthology.orchestra.run.smithi120.stdout:2166 trunc from 0xe3f9197 to 0x703d3dd 2024-09-18T01:08:45.520 INFO:teuthology.orchestra.run.smithi120.stdout:2167 punch from 0x6217f4f to 0x621f70d, (0x77be bytes) 2024-09-18T01:08:45.521 INFO:teuthology.orchestra.run.smithi120.stdout:2169 trunc from 0x703d3dd to 0x7c30ba4 2024-09-18T01:08:45.527 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.527 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.537 INFO:teuthology.orchestra.run.smithi120.stdout:2172 write 0xb0dab17 thru 0xb0e7034 (0xc51e bytes) 2024-09-18T01:08:45.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:45.548 INFO:teuthology.orchestra.run.smithi120.stdout:2173 trunc from 0xb0e7035 to 0x61bf514 2024-09-18T01:08:45.563 INFO:teuthology.orchestra.run.smithi120.stdout:2174 punch from 0x19b822d to 0x19c0986, (0x8759 bytes) 2024-09-18T01:08:45.563 INFO:teuthology.orchestra.run.smithi120.stdout:2175 read 0x3ed3e26 thru 0x3edf11a (0xb2f5 bytes) 2024-09-18T01:08:45.563 INFO:teuthology.orchestra.run.smithi120.stdout:2177 read 0x38168e9 thru 0x3825ae6 (0xf1fe bytes) 2024-09-18T01:08:45.563 INFO:teuthology.orchestra.run.smithi120.stdout:2178 read 0x139b0fe thru 0x139be4d (0xd50 bytes) 2024-09-18T01:08:45.564 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:45.564 INFO:teuthology.orchestra.run.smithi120.stdout:2183 read 0x3813e02 thru 0x381dd25 (0x9f24 bytes) 2024-09-18T01:08:45.570 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:08:45.570 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:45.574 INFO:teuthology.orchestra.run.smithi120.stdout:2184 write 0x903deca thru 0x9049fda (0xc111 bytes) 2024-09-18T01:08:45.576 INFO:teuthology.orchestra.run.smithi120.stdout:2185 clone 23 order 18 su 131072 sc 9 2024-09-18T01:08:45.597 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:45.597+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:45.645 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.645 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.654 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:45.698 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:45.697+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:45.757 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.757 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.766 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:45.801 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:45.800+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:45.858 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:08:45.875 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.875 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:45.915 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:45.914+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:45.989 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:45.989 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:45.998 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:46.042 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:46.041+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:46.101 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.101 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.104 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:46.103+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:46.109 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:46.205 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:46.204+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:46.253 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.254 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:46.306 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:46.305+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:46.406 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.406 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.406 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:46.405+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:46.414 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:46.508 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:46.507+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:46.556 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.557 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.565 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:46.609 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:46.608+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:46.668 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.668 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:46.709 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:46.708+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:46.810 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:46.809+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:46.821 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:46.821 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:46.830 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:46.914 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:46.913+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:47.012 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:47.011+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:47.016 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:47.017 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:47.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:47.113 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:47.112+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:47.144 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:47.144 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:47.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:47.238 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:47.237+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:47.298 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:47.298 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:47.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:47.348 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:47.347+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:47.416 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:47.415+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:47.450 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:47.450 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:47.454 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:47.459 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:47.516 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:47.516+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:47.577 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:47.577 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:47.594 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:47.594 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:47.605 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:47.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:47.605 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:47.605 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:47.610 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:47.610 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:47.614 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:47.617 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:47.616+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:47.627 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:47.627 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:47.644 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:47.645 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:47.661 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:47.678 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:47.678 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:47.695 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:47.695 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:47.718 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:47.717+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:47.719 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:47.719 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:47.719 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:47.719 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:47.719 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:47.727 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:47.819 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:47.818+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:47.874 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2178 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1540096~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.878 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697315918000001, 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023733608, 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2178 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178219+0000", 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178221+0000", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7799999999999999e-06 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178227+0000", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8479999999999997e-06 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178230+0000", 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6320000000000001e-06 2024-09-18T01:08:47.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201785+0000", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023554525 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201861+0000", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6265000000000002e-05 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201952+0000", 2024-09-18T01:08:47.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1161000000000004e-05 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2179 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1671168~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:47.883 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697299349999994, 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023913014999999999, 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2179 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178235+0000", 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178236+0000", 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178239+0000", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.988e-06 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178245+0000", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178248+0000", 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1609999999999999e-06 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.201982+0000", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023734149999999999 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202053+0000", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1341999999999996e-05 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202148+0000", 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4934999999999998e-05 2024-09-18T01:08:47.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2180 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1802240~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697283163999998, 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024101483999999999, 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2180 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:47.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178252+0000", 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178253+0000", 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.088e-06 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178257+0000", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5990000000000002e-06 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178259+0000", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6340000000000001e-06 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202178+0000", 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023918828 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202256+0000", 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7624000000000001e-05 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202353+0000", 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7368999999999996e-05 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.892 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2181 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 1933312~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697271595000004, 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024278860999999999, 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2181 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178263+0000", 2024-09-18T01:08:47.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178264+0000", 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4600000000000003e-07 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178267+0000", 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178270+0000", 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202386+0000", 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024115943000000001 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202457+0000", 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1094999999999996e-05 2024-09-18T01:08:47.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202542+0000", 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5271000000000004e-05 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2182 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2064384~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697262229000003, 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024448285, 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2182 2024-09-18T01:08:47.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178272+0000", 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178273+0000", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178274+0000", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7000000000000003e-07 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178277+0000", 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8890000000000002e-06 2024-09-18T01:08:47.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178280+0000", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1439999999999999e-06 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202570+0000", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024290683 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202638+0000", 2024-09-18T01:08:47.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7403999999999994e-05 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202721+0000", 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2987999999999999e-05 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2183 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2195456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:47.903 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697250045999994, 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024639942000000001, 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2183 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178285+0000", 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178287+0000", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.668e-06 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178292+0000", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.865e-06 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178295+0000", 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5980000000000001e-06 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202769+0000", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024474005 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202841+0000", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1495999999999994e-05 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202925+0000", 2024-09-18T01:08:47.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3967000000000004e-05 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2184 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2326528~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697234351000006, 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024804439000000001, 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2184 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178300+0000", 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178301+0000", 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178302+0000", 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178307+0000", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178312+0000", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1899999999999997e-06 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.202953+0000", 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024641562999999998 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203021+0000", 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7686000000000002e-05 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203105+0000", 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3941000000000002e-05 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.912 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2185 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2457600~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697217197000001, 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024971559000000001, 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2185 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178318+0000", 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.67e-07 2024-09-18T01:08:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178319+0000", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2449999999999999e-06 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178329+0000", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6150000000000003e-06 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178333+0000", 2024-09-18T01:08:47.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5449999999999997e-06 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203134+0000", 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024800727000000002 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203207+0000", 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2640999999999998e-05 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203289+0000", 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2519000000000001e-05 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2186 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697195031000007, 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025135481000000001, 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2186 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178340+0000", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178344+0000", 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:08:47.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178349+0000", 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178353+0000", 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9650000000000002e-06 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203317+0000", 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.024964534 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203381+0000", 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3714999999999996e-05 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203475+0000", 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4191999999999995e-05 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.921 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2187 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2719744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697178581000003, 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025326100000000001, 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2187 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178356+0000", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.48e-07 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178357+0000", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178360+0000", 2024-09-18T01:08:47.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-06 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178363+0000", 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3879999999999998e-06 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203519+0000", 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025156635 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203594+0000", 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4270999999999994e-05 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203682+0000", 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8505999999999996e-05 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2188 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2850816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.926 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697169126999995, 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025538795999999999, 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2188 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.927 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178366+0000", 2024-09-18T01:08:47.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178367+0000", 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6500000000000001e-06 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178370+0000", 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0110000000000001e-06 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178373+0000", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203724+0000", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025350502 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203818+0000", 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203904+0000", 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6595999999999996e-05 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2189 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 2981888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697135075999995, 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025681513, 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2189 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178400+0000", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178402+0000", 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-09-18T01:08:47.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178407+0000", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3140000000000003e-06 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178413+0000", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7559999999999996e-06 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.203933+0000", 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025519968000000001 2024-09-18T01:08:47.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204001+0000", 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7980999999999998e-05 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204081+0000", 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.996e-05 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2190 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3112960~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697115351999997, 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025838252999999999, 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2190 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178419+0000", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178420+0000", 2024-09-18T01:08:47.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178422+0000", 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-06 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178427+0000", 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3279999999999996e-06 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178432+0000", 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9110000000000001e-06 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204106+0000", 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025674718999999999 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204175+0000", 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8454999999999994e-05 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204258+0000", 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2701999999999995e-05 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2191 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697097115000005, 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026771867000000001, 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2191 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178438+0000", 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.58e-07 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178440+0000", 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.832e-06 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178446+0000", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6760000000000001e-06 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178452+0000", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.434e-06 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204287+0000", 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.025834755000000001 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.204355+0000", 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8770999999999996e-05 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205209+0000", 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085404100000000004 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.944 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2192 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3375104~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697076539999998, 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026936413999999999, 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2192 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178458+0000", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178459+0000", 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:08:47.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178460+0000", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7549999999999999e-06 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178466+0000", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6999999999999996e-06 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178471+0000", 2024-09-18T01:08:47.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8330000000000001e-06 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205240+0000", 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026769566000000002 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205311+0000", 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0957000000000005e-05 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205395+0000", 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3227999999999997e-05 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2193 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3506176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697057313000002, 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027102502000000001, 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2193 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178477+0000", 2024-09-18T01:08:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178478+0000", 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178479+0000", 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7150000000000001e-06 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178486+0000", 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1369999999999999e-06 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178490+0000", 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.357e-06 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205422+0000", 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026931618000000001 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205492+0000", 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0024000000000006e-05 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205580+0000", 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2194 3.4 3:27ef45b0:::rbd_data.1176f5693ee0.0000000000000011:3 [read 3637248~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697037960000003, 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027307978, 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2194 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178497+0000", 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178499+0000", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178504+0000", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.507e-06 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178509+0000", 2024-09-18T01:08:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4750000000000004e-06 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205608+0000", 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027099364000000001 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205688+0000", 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9943000000000004e-05 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205805+0000", 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011654699999999999 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.697019304999998, 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.958 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:08:47.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.688237356000002, 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:08:47.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:08:47.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:08:47.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:47.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:50519 3.19 3:9a8c2274:::rbd_data.11761711706f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:47.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.456786247, 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026218866, 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 50519 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418748+0000", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418749+0000", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.85e-07 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418754+0000", 2024-09-18T01:08:47.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2910000000000001e-06 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418764+0000", 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418769+0000", 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1789999999999999e-06 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418814+0000", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4350000000000001e-05 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.418863+0000", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9264000000000001e-05 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.423079+0000", 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042156269999999996 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444883+0000", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.021804975000000001 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444896+0000", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2344000000000001e-05 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444929+0000", 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.269e-05 2024-09-18T01:08:47.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:25.444967+0000", 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8818e-05 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:47.976 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:47.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:47.978 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:47.976+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:47.991 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-18T01:08:48.079 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:48.078+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:48.107 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:48.108 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:48.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:48.180 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:48.180+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:48.224 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:48.224 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:48.224 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:48.224 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:48.224 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:48.233 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:48.244 INFO:teuthology.orchestra.run.smithi120.stdout:2186 trunc from 0x9049fdb to 0x4a14021 2024-09-18T01:08:48.281 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:48.280+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:48.302 INFO:teuthology.orchestra.run.smithi120.stdout:2187 punch from 0x477ee09 to 0x478e990, (0xfb87 bytes) 2024-09-18T01:08:48.302 INFO:teuthology.orchestra.run.smithi120.stdout:2188 read 0x286ea93 thru 0x2871e5d (0x33cb bytes) 2024-09-18T01:08:48.314 INFO:teuthology.orchestra.run.smithi120.stdout:2189 trunc from 0x4a14021 to 0xd77aaec 2024-09-18T01:08:48.322 INFO:teuthology.orchestra.run.smithi120.stdout:2190 punch from 0x82a65bc to 0x82b4a58, (0xe49c bytes) 2024-09-18T01:08:48.323 INFO:teuthology.orchestra.run.smithi120.stdout:2191 punch from 0xd67a3b0 to 0xd680816, (0x6466 bytes) 2024-09-18T01:08:48.323 INFO:teuthology.orchestra.run.smithi120.stdout:2192 read 0x32be850 thru 0x32c952d (0xacde bytes) 2024-09-18T01:08:48.330 INFO:teuthology.orchestra.run.smithi120.stdout:2193 trunc from 0xd77aaec to 0x6564bf9 2024-09-18T01:08:48.336 INFO:teuthology.orchestra.run.smithi120.stdout:2194 write 0x61fec31 thru 0x62023fb (0x37cb bytes) 2024-09-18T01:08:48.355 INFO:teuthology.orchestra.run.smithi120.stdout:2195 writesame 0xbc0c38c thru 0xbc11e4d data_size 0x1(0x5ac2 bytes) 2024-09-18T01:08:48.362 INFO:teuthology.orchestra.run.smithi120.stdout:2196 write 0xd4d5f5b thru 0xd4e3872 (0xd918 bytes) 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 89.430535042000002, 2024-09-18T01:08:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.302607002000002, 2024-09-18T01:08:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.252232942000006, 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.162339725999999, 2024-09-18T01:08:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.160743977999999, 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.159933358999993, 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:10053 3.9 e37/23 3:913c0ce6:::rbd_data.117611b60058.0000000000000013:head v 37'15, mlcod=31'13)", 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.776560548000006, 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037015279999999999, 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593699+0000", 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.593700+0000", 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6600000000000003e-07 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594057+0000", 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035778399999999999 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594070+0000", 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3005e-05 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594076+0000", 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8939999999999998e-06 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594131+0000", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5033e-05 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.594155+0000", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3397e-05 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597359+0000", 2024-09-18T01:08:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032046959999999999 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:43.597401+0000", 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1452999999999998e-05 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.0 epoch 38/23 rep_tid 156 entries 38'75 (0'0) error 3:00333f82:::rbd_data.117611b60058.0000000000000115:head by client.4470.0:13208 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'73)", 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.212960490999997, 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036854890000000001, 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157299+0000", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157300+0000", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157307+0000", 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6989999999999997e-06 2024-09-18T01:08:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157322+0000", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4878000000000001e-05 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157331+0000", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8720000000000008e-06 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.157386+0000", 2024-09-18T01:08:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5957e-05 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.160985+0000", 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035984609999999998 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.195199404999997, 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.192542701999997, 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.609895158999997, 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.195581699999998, 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.693882678000001, 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.692305791000003, 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.662558201000003, 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:48.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:48.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39798 3.4 e63/23 3:24947a13:::rbd_data.11762bc2f01c.0000000000000036:head v 63'121, mlcod=61'119)", 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.616915040000002, 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035653780000000001, 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.440 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753345+0000", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753346+0000", 2024-09-18T01:08:48.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.08e-06 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753426+0000", 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9610999999999994e-05 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753443+0000", 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7170999999999999e-05 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753451+0000", 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5350000000000004e-06 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753516+0000", 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4208999999999995e-05 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.753538+0000", 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2628000000000001e-05 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756846+0000", 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033078280000000001 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.756910+0000", 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4315999999999999e-05 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.444 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.507178715000002, 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:48.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:48.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.503525285000002, 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:48.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:52190 3.4 e82/23 3:2587a2b7:::rbd_data.11762afc950e.0000000000000022:head v 82'135, mlcod=77'133)", 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.767309095000002, 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038334850000000002, 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:48.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602975+0000", 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3629999999999999e-05 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602984+0000", 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0820000000000005e-06 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602992+0000", 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6920000000000002e-06 2024-09-18T01:08:48.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603100+0000", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107977 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603124+0000", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4544000000000001e-05 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606735+0000", 2024-09-18T01:08:48.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036103580000000001 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606784+0000", 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9697000000000002e-05 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:48.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.7878586100000007, 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:48.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:08:48.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:08:48.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:48.463 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:48.464 INFO:teuthology.orchestra.run.smithi120.stdout:2197 read 0x1f9beda thru 0x1fa5961 (0x9a88 bytes) 2024-09-18T01:08:48.464 INFO:teuthology.orchestra.run.smithi120.stdout:2198 read 0x2996ff7 thru 0x299a705 (0x370f bytes) 2024-09-18T01:08:48.464 INFO:teuthology.orchestra.run.smithi120.stdout:2201 write 0x8f35b99 thru 0x8f3cf7d (0x73e5 bytes) 2024-09-18T01:08:48.464 INFO:teuthology.orchestra.run.smithi120.stdout:2202 punch from 0xaccdeb5 to 0xacd5070, (0x71bb bytes) 2024-09-18T01:08:48.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:48.466 INFO:teuthology.orchestra.run.smithi120.stdout:2203 trunc from 0xd4e3873 to 0x83f348b 2024-09-18T01:08:48.466 INFO:teuthology.orchestra.run.smithi120.stdout:2206 trunc from 0x83f348b to 0x11c7354 2024-09-18T01:08:48.466 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:48.463+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:48.468 INFO:teuthology.orchestra.run.smithi120.stdout:2207 write 0x2654024 thru 0x2658ff1 (0x4fce bytes) 2024-09-18T01:08:48.475 INFO:teuthology.orchestra.run.smithi120.stdout:2209 writesame 0x35f3fc4 thru 0x35fef13 data_size 0xb0(0xaf50 bytes) 2024-09-18T01:08:48.476 INFO:teuthology.orchestra.run.smithi120.stdout:2210 trunc from 0x35fef14 to 0x2b62092 2024-09-18T01:08:48.481 INFO:teuthology.orchestra.run.smithi120.stdout:2211 read 0x2a23ac9 thru 0x2a30b7b (0xd0b3 bytes) 2024-09-18T01:08:48.481 INFO:teuthology.orchestra.run.smithi120.stdout:2212 read 0xd3c17e thru 0xd47a5a (0xb8dd bytes) 2024-09-18T01:08:48.482 INFO:teuthology.orchestra.run.smithi120.stdout:2213 punch from 0x1b5f091 to 0x1b686ab, (0x961a bytes) 2024-09-18T01:08:48.501 INFO:teuthology.orchestra.run.smithi120.stdout:2214 writesame 0xeada056 thru 0xeada403 data_size 0x3(0x3ae bytes) 2024-09-18T01:08:48.501 INFO:teuthology.orchestra.run.smithi120.stdout:2218 write 0xbe9ecf8 thru 0xbead1c8 (0xe4d1 bytes) 2024-09-18T01:08:48.508 INFO:teuthology.orchestra.run.smithi120.stdout:2220 read 0x91f3c9f thru 0x91f58e5 (0x1c47 bytes) 2024-09-18T01:08:48.508 INFO:teuthology.orchestra.run.smithi120.stdout:2221 read 0x53768dc thru 0x537c9f2 (0x6117 bytes) 2024-09-18T01:08:48.509 INFO:teuthology.orchestra.run.smithi120.stdout:2222 trunc from 0xeada404 to 0xd6caefd 2024-09-18T01:08:48.515 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.515 INFO:teuthology.orchestra.run.smithi120.stdout:2225 punch from 0x3e07dd8 to 0x3e0baee, (0x3d16 bytes) 2024-09-18T01:08:48.515 INFO:teuthology.orchestra.run.smithi120.stdout:2226 write 0x5100162 thru 0x5109710 (0x95af bytes) 2024-09-18T01:08:48.518 INFO:teuthology.orchestra.run.smithi120.stdout:2227 write 0x3f6dbe0 thru 0x3f7999b (0xbdbc bytes) 2024-09-18T01:08:48.533 INFO:teuthology.orchestra.run.smithi120.stdout:2228 trunc from 0xd6caefd to 0x2f09dad 2024-09-18T01:08:48.546 INFO:teuthology.orchestra.run.smithi120.stdout:2229 punch from 0x1e92fb8 to 0x1e9b110, (0x8158 bytes) 2024-09-18T01:08:48.546 INFO:teuthology.orchestra.run.smithi120.stdout:2230 trunc from 0x2f09dad to 0x33bb3ae 2024-09-18T01:08:48.558 INFO:teuthology.orchestra.run.smithi120.stdout:2232 write 0x4cd0e1b thru 0x4cd2171 (0x1357 bytes) 2024-09-18T01:08:48.565 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:48.564+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:48.572 INFO:teuthology.orchestra.run.smithi120.stdout:2234 write 0x8a22088 thru 0x8a31896 (0xf80f bytes) 2024-09-18T01:08:48.575 INFO:teuthology.orchestra.run.smithi120.stdout:2235 read 0x498c1da thru 0x499a8e6 (0xe70d bytes) 2024-09-18T01:08:48.575 INFO:teuthology.orchestra.run.smithi120.stdout:2236 punch from 0x43d7590 to 0x43e0117, (0x8b87 bytes) 2024-09-18T01:08:48.586 INFO:teuthology.orchestra.run.smithi120.stdout:2237 write 0xcde886f thru 0xcded8eb (0x507d bytes) 2024-09-18T01:08:48.587 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:48.587 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:48.589 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.589 INFO:teuthology.orchestra.run.smithi120.stdout:2239 read 0x642dbdd thru 0x64312e7 (0x370b bytes) 2024-09-18T01:08:48.589 INFO:teuthology.orchestra.run.smithi120.stdout:2240 write 0x787e114 thru 0x7888b96 (0xaa83 bytes) 2024-09-18T01:08:48.593 INFO:teuthology.orchestra.run.smithi120.stdout:2242 write 0x64e24c2 thru 0x64e8f88 (0x6ac7 bytes) 2024-09-18T01:08:48.596 INFO:teuthology.orchestra.run.smithi120.stdout:2243 read 0x8aef9e4 thru 0x8afd388 (0xd9a5 bytes) 2024-09-18T01:08:48.596 INFO:teuthology.orchestra.run.smithi120.stdout:2244 punch from 0x595e1d2 to 0x5967c34, (0x9a62 bytes) 2024-09-18T01:08:48.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:48.603 INFO:teuthology.orchestra.run.smithi120.stdout:2245 trunc from 0xcded8ec to 0x6ecfaba 2024-09-18T01:08:48.622 INFO:teuthology.orchestra.run.smithi120.stdout:2246 trunc from 0x6ecfaba to 0xdf6af3b 2024-09-18T01:08:48.628 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.628 INFO:teuthology.orchestra.run.smithi120.stdout:2250 write 0x50653d thru 0x50bab6 (0x557a bytes) 2024-09-18T01:08:48.636 INFO:teuthology.orchestra.run.smithi120.stdout:2251 trunc from 0xdf6af3b to 0x995dcaa 2024-09-18T01:08:48.649 INFO:teuthology.orchestra.run.smithi120.stdout:2252 write 0xa4bec64 thru 0xa4cb01f (0xc3bc bytes) 2024-09-18T01:08:48.658 INFO:teuthology.orchestra.run.smithi120.stdout:2253 writesame 0x5516fe8 thru 0x551afa0 data_size 0x1(0x3fb9 bytes) 2024-09-18T01:08:48.666 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:48.665+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:48.669 INFO:teuthology.orchestra.run.smithi120.stdout:2254 write 0x3b8897d thru 0x3b97cc9 (0xf34d bytes) 2024-09-18T01:08:48.677 INFO:teuthology.orchestra.run.smithi120.stdout:2255 trunc from 0xa4cb020 to 0x6c170ad 2024-09-18T01:08:48.687 INFO:teuthology.orchestra.run.smithi120.stdout:2256 punch from 0xf87448 to 0xf87d9c, (0x954 bytes) 2024-09-18T01:08:48.698 INFO:teuthology.orchestra.run.smithi120.stdout:2257 writesame 0xb671a9f thru 0xb67fa0b data_size 0x1(0xdf6d bytes) 2024-09-18T01:08:48.698 INFO:teuthology.orchestra.run.smithi120.stdout:2258 read 0x5770430 thru 0x5776362 (0x5f33 bytes) 2024-09-18T01:08:48.701 INFO:teuthology.orchestra.run.smithi120.stdout:2259 trunc from 0xb67fa0c to 0x88ad10d 2024-09-18T01:08:48.717 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:48.717 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:48.717 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:48.717 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:48.717 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:48.719 INFO:teuthology.orchestra.run.smithi120.stdout:2261 write 0xe86aec5 thru 0xe86b8c3 (0x9ff bytes) 2024-09-18T01:08:48.722 INFO:teuthology.orchestra.run.smithi120.stdout:2263 read 0xcc978ee thru 0xcca52ec (0xd9ff bytes) 2024-09-18T01:08:48.724 INFO:teuthology.orchestra.run.smithi120.stdout:2264 trunc from 0xe86b8c4 to 0xc83e908 2024-09-18T01:08:48.728 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:48.733 INFO:teuthology.orchestra.run.smithi120.stdout:2266 trunc from 0xc83e908 to 0xcede469 2024-09-18T01:08:48.739 INFO:teuthology.orchestra.run.smithi120.stdout:2267 trunc from 0xcede469 to 0xd8fbf76 2024-09-18T01:08:48.744 INFO:teuthology.orchestra.run.smithi120.stdout:2268 read 0xd817077 thru 0xd8199fc (0x2986 bytes) 2024-09-18T01:08:48.745 INFO:teuthology.orchestra.run.smithi120.stdout:2269 writesame 0xd14f24e thru 0xd156f65 data_size 0x4(0x7d18 bytes) 2024-09-18T01:08:48.750 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.751 INFO:teuthology.orchestra.run.smithi120.stdout:2273 read 0xcbfbd59 thru 0xcbfea85 (0x2d2d bytes) 2024-09-18T01:08:48.751 INFO:teuthology.orchestra.run.smithi120.stdout:2275 writesame 0x7fff7c7 thru 0x800365b data_size 0x1(0x3e95 bytes) 2024-09-18T01:08:48.762 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.762 INFO:teuthology.orchestra.run.smithi120.stdout:2279 read 0xd496132 thru 0xd49edd4 (0x8ca3 bytes) 2024-09-18T01:08:48.762 INFO:teuthology.orchestra.run.smithi120.stdout:2280 read 0x19518bc thru 0x195f035 (0xd77a bytes) 2024-09-18T01:08:48.762 INFO:teuthology.orchestra.run.smithi120.stdout:2281 punch from 0x603840b to 0x603d7d9, (0x53ce bytes) 2024-09-18T01:08:48.762 INFO:teuthology.orchestra.run.smithi120.stdout:2282 write 0x44380a9 thru 0x443f76e (0x76c6 bytes) 2024-09-18T01:08:48.765 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.765 INFO:teuthology.orchestra.run.smithi120.stdout:2284 read 0xae48071 thru 0xae522f2 (0xa282 bytes) 2024-09-18T01:08:48.765 INFO:teuthology.orchestra.run.smithi120.stdout:2285 punch from 0x8f9f0b7 to 0x8fac1f4, (0xd13d bytes) 2024-09-18T01:08:48.765 INFO:teuthology.orchestra.run.smithi120.stdout:2286 write 0x1eb6522 thru 0x1ec3259 (0xcd38 bytes) 2024-09-18T01:08:48.767 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:48.767+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:48.768 INFO:teuthology.orchestra.run.smithi120.stdout:2287 write 0x9d09e51 thru 0x9d13481 (0x9631 bytes) 2024-09-18T01:08:48.772 INFO:teuthology.orchestra.run.smithi120.stdout:2288 punch from 0x55d4ff3 to 0x55dccad, (0x7cba bytes) 2024-09-18T01:08:48.778 INFO:teuthology.orchestra.run.smithi120.stdout:2289 trunc from 0xd8fbf76 to 0x7a43ea6 2024-09-18T01:08:48.789 INFO:teuthology.orchestra.run.smithi120.stdout:2291 read 0x3021950 thru 0x3023968 (0x2019 bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:2292 read 0x6d38020 thru 0x6d41148 (0x9129 bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:2293 punch from 0x404445e to 0x40453c1, (0xf63 bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:2294 read 0x5cfc0da thru 0x5d01682 (0x55a9 bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:2296 punch from 0x565027c to 0x56506e9, (0x46d bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:2299 read 0x5e5c32f thru 0x5e64906 (0x85d8 bytes) 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.790 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.793 INFO:teuthology.orchestra.run.smithi120.stdout:2303 trunc from 0x7a43ea6 to 0xa7d379f 2024-09-18T01:08:48.798 INFO:teuthology.orchestra.run.smithi120.stdout:2304 read 0x5c8b467 thru 0x5c905f9 (0x5193 bytes) 2024-09-18T01:08:48.798 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:48.808 INFO:teuthology.orchestra.run.smithi120.stdout:2306 write 0xe49a29e thru 0xe4a927d (0xefe0 bytes) 2024-09-18T01:08:48.823 INFO:teuthology.orchestra.run.smithi120.stdout:2309 trunc from 0xe4a927e to 0xa400ccc 2024-09-18T01:08:48.834 INFO:teuthology.orchestra.run.smithi120.stdout:2311 writesame 0x17687cd thru 0x176aba9 data_size 0x1(0x23dd bytes) 2024-09-18T01:08:49.038 INFO:teuthology.orchestra.run.smithi120.stdout:2314 writesame 0x3616b18 thru 0x36259c7 data_size 0x1(0xeeb0 bytes) 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.039 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38805 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2424832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280352037999997, 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244350999999999, 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38805 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:49.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569048+0000", 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569057+0000", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4530000000000008e-06 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569060+0000", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0170000000000001e-06 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611025+0000", 2024-09-18T01:08:49.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041964317000000001 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611073+0000", 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8037000000000001e-05 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611288+0000", 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215596 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38806 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280332213000001, 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042508381999999997, 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38806 2024-09-18T01:08:49.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:49.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4400000000000001e-07 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569065+0000", 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.206e-06 2024-09-18T01:08:49.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569068+0000", 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2890000000000002e-06 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569071+0000", 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611316+0000", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244918999999999 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611360+0000", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4239000000000001e-05 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611572+0000", 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212186 2024-09-18T01:08:49.049 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38807 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280322460000001, 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042802420000000001, 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.050 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38807 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569074+0000", 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569075+0000", 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569078+0000", 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9749999999999999e-06 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569080+0000", 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611602+0000", 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042521681999999998 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611650+0000", 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8179000000000001e-05 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611876+0000", 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022609299999999999 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38808 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280312610999999, 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043110682999999997, 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38808 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.055 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569084+0000", 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569085+0000", 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:08:49.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569088+0000", 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569090+0000", 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3120000000000001e-06 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611901+0000", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042810991999999999 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611955+0000", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3983000000000001e-05 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612194+0000", 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239182 2024-09-18T01:08:49.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38809 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 475136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280294286, 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043389637000000002, 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38809 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569122+0000", 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0254999999999998e-05 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569127+0000", 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569130+0000", 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.016e-06 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612224+0000", 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043094406000000002 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612275+0000", 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1054000000000003e-05 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612491+0000", 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215987 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38810 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280261439999997, 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043652712000000003, 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38810 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569135+0000", 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569136+0000", 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.083e-06 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569141+0000", 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569143+0000", 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3369999999999998e-06 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612521+0000", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043377538 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612566+0000", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5485000000000002e-05 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612787+0000", 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221158 2024-09-18T01:08:49.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38811 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280250158999998, 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043930856999999997, 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38811 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569147+0000", 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0449999999999999e-06 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569150+0000", 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569152+0000", 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612830+0000", 2024-09-18T01:08:49.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043677597999999998 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612880+0000", 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0219999999999997e-05 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613077+0000", 2024-09-18T01:08:49.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019646600000000001 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38812 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.28024095, 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044183260000000002, 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38812 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569156+0000", 2024-09-18T01:08:49.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999991e-07 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569160+0000", 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3759999999999999e-06 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569162+0000", 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613107+0000", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043945166000000001 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613153+0000", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613338+0000", 2024-09-18T01:08:49.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018564800000000001 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38813 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1130496~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280231481999998, 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044519186000000002, 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38813 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569165+0000", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569166+0000", 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2399999999999996e-07 2024-09-18T01:08:49.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569168+0000", 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9229999999999998e-06 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569178+0000", 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6479999999999998e-06 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613370+0000", 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044191800000000003 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613416+0000", 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6431999999999999e-05 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613684+0000", 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026725300000000001 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.084 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38814 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280214651000001, 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044848796000000003, 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38814 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569183+0000", 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.096e-06 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569187+0000", 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569191+0000", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1930000000000002e-06 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613735+0000", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044543886999999997 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613784+0000", 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9858999999999998e-05 2024-09-18T01:08:49.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614030+0000", 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024569700000000002 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38815 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1458176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280202539999998, 2024-09-18T01:08:49.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045101439, 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38815 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569194+0000", 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569195+0000", 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0389999999999999e-06 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569198+0000", 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9210000000000001e-06 2024-09-18T01:08:49.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569200+0000", 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.367e-06 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614061+0000", 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044861375000000002 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614104+0000", 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3142000000000001e-05 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614295+0000", 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190383 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.096 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38816 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1622016~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280193429999997, 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045464629999999999, 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38816 2024-09-18T01:08:49.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569203+0000", 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569206+0000", 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.754e-06 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569211+0000", 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.109e-06 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569213+0000", 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.379e-06 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614331+0000", 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045118395999999998 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614380+0000", 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614667+0000", 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028724000000000002 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38817 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1785856~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280180037000001, 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045799302, 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38817 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569217+0000", 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.035e-06 2024-09-18T01:08:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569221+0000", 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.839e-06 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569223+0000", 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3300000000000001e-06 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614711+0000", 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045487811000000003 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614772+0000", 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0729000000000001e-05 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615015+0000", 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024334399999999999 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38818 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280169987000001, 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046120525000000002, 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38818 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569230+0000", 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0260000000000001e-06 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569234+0000", 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569237+0000", 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3750000000000001e-06 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615046+0000", 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045809435000000003 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615096+0000", 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9654999999999998e-05 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615346+0000", 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025073700000000001 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38819 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2113536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280154199999998, 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046434472999999997, 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38819 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569243+0000", 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.291e-06 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569247+0000", 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.031e-06 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569250+0000", 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615395+0000", 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046145046000000002 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615437+0000", 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2407999999999998e-05 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615676+0000", 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239137 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38820 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2277376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280143301999999, 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046745223000000002, 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38820 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569254+0000", 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-07 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569258+0000", 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6489999999999999e-06 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569261+0000", 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.322e-06 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615704+0000", 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046442986999999998 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615764+0000", 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9871000000000003e-05 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615998+0000", 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023420099999999999 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38821 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2441216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280132449999996, 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047015754999999999, 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38821 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569264+0000", 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569265+0000", 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8599999999999996e-07 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569269+0000", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8800000000000001e-06 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569271+0000", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616017+0000", 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046746593000000003 2024-09-18T01:08:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616061+0000", 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3894999999999999e-05 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616279+0000", 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000217875 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38822 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.280122378000002, 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047344570000000002, 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38822 2024-09-18T01:08:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569275+0000", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.026e-06 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569278+0000", 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999999e-06 2024-09-18T01:08:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569284+0000", 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.071e-06 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616318+0000", 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047033859999999997 2024-09-18T01:08:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616374+0000", 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5636000000000001e-05 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616618+0000", 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024412699999999999 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38823 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.276145804999999, 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045935959999999998, 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38823 2024-09-18T01:08:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573251+0000", 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573255+0000", 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2520000000000001e-06 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573266+0000", 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0958000000000001e-05 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573271+0000", 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1229999999999999e-06 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616299+0000", 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043027351999999998 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616351+0000", 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2284000000000003e-05 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616681+0000", 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033019400000000002 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619078+0000", 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023970749999999998 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619093+0000", 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4338999999999999e-05 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619143+0000", 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0346000000000002e-05 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619186+0000", 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3226999999999998e-05 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38824 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.270806456000003, 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042771339999999998, 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38824 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578590+0000", 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578594+0000", 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0609999999999997e-06 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578605+0000", 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0477e-05 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578611+0000", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9540000000000001e-06 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617084+0000", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.038473680000000003 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617135+0000", 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0139000000000002e-05 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617316+0000", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181082 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621302+0000", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039860210000000002 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621309+0000", 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6490000000000008e-06 2024-09-18T01:08:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621336+0000", 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6828000000000001e-05 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621361+0000", 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4621999999999999e-05 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:49.146 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:49.146 INFO:teuthology.orchestra.run.smithi120.stdout:2315 punch from 0x6aa769a to 0x6aa9ef4, (0x285a bytes) 2024-09-18T01:08:49.146 INFO:teuthology.orchestra.run.smithi120.stdout:2316 write 0x1b36ae4 thru 0x1b39a52 (0x2f6f bytes) 2024-09-18T01:08:49.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:49.148 INFO:teuthology.orchestra.run.smithi120.stdout:2317 punch from 0x3c5ca39 to 0x3c5ddff, (0x13c6 bytes) 2024-09-18T01:08:49.148 INFO:teuthology.orchestra.run.smithi120.stdout:2318 write 0x4752da4 thru 0x47559eb (0x2c48 bytes) 2024-09-18T01:08:49.148 INFO:teuthology.orchestra.run.smithi120.stdout:2319 writesame 0x946c670 thru 0x9479eff data_size 0x63(0xd890 bytes) 2024-09-18T01:08:49.149 INFO:teuthology.orchestra.run.smithi120.stdout:2320 writesame 0x993630a thru 0x99397ee data_size 0x1(0x34e5 bytes) 2024-09-18T01:08:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:08:49.150 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:49.037+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:2321 write 0xa6f3c55 thru 0xa6f6155 (0x2501 bytes) 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:2326 punch from 0x30b8557 to 0x30c1c7f, (0x9728 bytes) 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:2327 write 0x70af646 thru 0x70b98e0 (0xa29b bytes) 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:2329 write 0x90c347b thru 0x90cfa33 (0xc5b9 bytes) 2024-09-18T01:08:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2331 write 0x28a9482 thru 0x28b2e1f (0x999e bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2332 punch from 0x120c4db to 0x121a239, (0xdd5e bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2333 writesame 0x1a9b99f thru 0x1aa2829 data_size 0x1(0x6e8b bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2334 write 0x907d98a thru 0x908ad20 (0xd397 bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2335 write 0x5ccab40 thru 0x5ccdf94 (0x3455 bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2336 read 0x2b83b27 thru 0x2b8fe11 (0xc2eb bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2338 trunc from 0xa6f6156 to 0x8998416 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2342 write 0xaaaf7b4 thru 0xaab547a (0x5cc7 bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2343 write 0x6881dc2 thru 0x6888878 (0x6ab7 bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:2344 read 0x3dc6ce9 thru 0x3dcd1d3 (0x64eb bytes) 2024-09-18T01:08:49.156 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2348 read 0x21a541f thru 0x21b2b8e (0xd770 bytes) 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2349 write 0x69475f1 thru 0x694d48a (0x5e9a bytes) 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2353 trunc from 0xaab547b to 0x401d4a9 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2354 writesame 0x57a8ee1 thru 0x57b6ed7 data_size 0x1(0xdff7 bytes) 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2356 writesame 0x6cc4a30 thru 0x6cc9d81 data_size 0x12(0x5352 bytes) 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2361 read 0x5f84857 thru 0x5f893d6 (0x4b80 bytes) 2024-09-18T01:08:49.157 INFO:teuthology.orchestra.run.smithi120.stdout:2362 writesame 0x3745405 thru 0x3749b8b data_size 0x1(0x4787 bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2364 punch from 0x2402176 to 0x24120cc, (0xff56 bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2365 write 0x36cb732 thru 0x36cbe42 (0x711 bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2366 read 0x5226d4a thru 0x522ace5 (0x3f9c bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2367 read 0x6ad5076 thru 0x6adf993 (0xa91e bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2369 punch from 0x16df789 to 0x16e797b, (0x81f2 bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2370 read 0x4916f3a thru 0x4925e23 (0xeeea bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2372 read 0x4bf198d thru 0x4bfe3dc (0xca50 bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2373 trunc from 0x6cc9d82 to 0x55564dc 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2375 writesame 0xe42cdfd thru 0xe431c27 data_size 0x1(0x4e2b bytes) 2024-09-18T01:08:49.158 INFO:teuthology.orchestra.run.smithi120.stdout:2376 punch from 0xad09407 to 0xad17879, (0xe472 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2378 writesame 0xe6da27c thru 0xe6e0e80 data_size 0x1(0x6c05 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2380 read 0x3819643 thru 0x381e34b (0x4d09 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2381 trunc from 0xe6e0e81 to 0xcdb6cf1 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2383 trunc from 0xcdb6cf1 to 0x21264c4 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2386 writesame 0xa26474d thru 0xa268ee9 data_size 0x3f(0x479d bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2388 write 0x61e192d thru 0x61e6517 (0x4beb bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2389 write 0x74edfff thru 0x74fc78c (0xe78e bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2390 punch from 0x2ccb4ee to 0x2cda4dd, (0xefef bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2391 punch from 0x1cdc4e2 to 0x1cdc8fa, (0x418 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2392 write 0xdfec4ff thru 0xdff256d (0x606f bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2393 write 0xe6d44e4 thru 0xe6d96b5 (0x51d2 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2395 write 0x2604e38 thru 0x260a856 (0x5a1f bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2397 read 0x4714ae6 thru 0x471968a (0x4ba5 bytes) 2024-09-18T01:08:49.159 INFO:teuthology.orchestra.run.smithi120.stdout:2398 clone 24 order 25 su 524288 sc 9 2024-09-18T01:08:49.189 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:49.188+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:49.251 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:49.250+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:49.253 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:49.253 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:49.253 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:49.253 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:49.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:49.352 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:49.351+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:49.366 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:49.366 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:49.366 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:49.366 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:49.366 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:49.375 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:49.460 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:49.459+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:49.520 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:49.520 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:49.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:49.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:49.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.926540618999994, 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:49.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.918208946999997, 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:49.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.914868092999995, 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:49.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.887413264000003, 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:49.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:49.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.853447947000006, 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:49.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.772480361000007, 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:49.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.772458522999997, 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:49.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.882113222999998, 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:49.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.880570358, 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.880549201000001, 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:49.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.880530387, 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:49.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.427012210999997, 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:49.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:49.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.426998290999997, 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:49.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:49.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 51.950452227, 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:49.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:49.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:49.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 51.795260368000001, 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.589 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:49.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.858254369000001, 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.596 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:49.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.794303855999999, 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:49.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:49.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:49.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.656560253999999, 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:49.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:49.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:58523 3.1a 3:5a42e52b:::rbd_data.11768408e4d7.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.055835525, 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014664071000000001, 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 58523 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462166+0000", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8700000000000005e-07 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462171+0000", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2939999999999999e-06 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462181+0000", 2024-09-18T01:08:49.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0908e-05 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462187+0000", 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462245+0000", 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8279999999999998e-05 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462294+0000", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8729000000000001e-05 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.465104+0000", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028099100000000001 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476746+0000", 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011641979 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476760+0000", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4087000000000001e-05 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476793+0000", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769999999999997e-05 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476830+0000", 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6171000000000003e-05 2024-09-18T01:08:49.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:61531 3.1a 3:5f2ef990:::rbd_data.117649d3c4d0.00000000000000d8:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.67573735499999998, 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016342980999999999, 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:49.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 61531 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:49.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842265+0000", 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7599999999999996e-07 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842269+0000", 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9150000000000004e-06 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:49.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842281+0000", 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1270000000000001e-05 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842286+0000", 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842325+0000", 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9431000000000002e-05 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842419+0000", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3338000000000007e-05 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.845634+0000", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032156099999999998 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858476+0000", 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012841491 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858494+0000", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8533e-05 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858555+0000", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0372999999999999e-05 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858607+0000", 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1861999999999999e-05 2024-09-18T01:08:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:49.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:49.626 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:49.627 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:49.628 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:49.626+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:49.707 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:49.751 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:49.750+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:49.771 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:49.771 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:49.780 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:49.827 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:49.827 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:49.828 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:49.827+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:49.843 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:49.843 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:49.859 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:49.876 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:49.885 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:49.885 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:49.885 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:49.885 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:49.885 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:49.892 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:49.892 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:49.894 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:49.909 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:49.925 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:49.929 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:49.928+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:49.943 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:50.030 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:50.029+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:50.037 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.037 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.046 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:50.131 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:50.130+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:50.189 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.189 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.198 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:50.232 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:50.231+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:50.333 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:50.332+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:50.341 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.341 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.350 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:50.435 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:50.434+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:50.493 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.493 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:50.543 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:50.542+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:50.582 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:50.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:08:50.636 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:50.635+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:50.672 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.672 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.681 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:50.766 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:50.765+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:50.826 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.826 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.835 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:50.876 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:50.875+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:50.939 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:50.938+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:50.978 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:50.978 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:50.988 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:51.040 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:51.039+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:51.091 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.091 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.100 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:51.141 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:51.140+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:51.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:08:51.241 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:51.240+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:51.245 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.245 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.254 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:51.342 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:51.340+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:51.399 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.399 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.408 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:51.452 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:51.451+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:51.512 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.513 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.521 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:51.544 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:51.543+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:51.563 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:08:51.564 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:08:51.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:08:51.645 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:51.644+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:51.673 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.673 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.683 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:51.727 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-18T01:08:51.748 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:51.745+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:51.811 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.811 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.823 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:51.864 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:51.863+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:51.903 INFO:teuthology.orchestra.run.smithi120.stdout:2400 write 0xb0253a7 thru 0xb02ac71 (0x58cb bytes) 2024-09-18T01:08:51.909 INFO:teuthology.orchestra.run.smithi120.stdout:2401 read 0x5e18d64 thru 0x5e23b76 (0xae13 bytes) 2024-09-18T01:08:51.909 INFO:teuthology.orchestra.run.smithi120.stdout:2403 write 0x6871d0e thru 0x687a2e2 (0x85d5 bytes) 2024-09-18T01:08:51.911 INFO:teuthology.orchestra.run.smithi120.stdout:2404 punch from 0x970214b to 0x970c8ec, (0xa7a1 bytes) 2024-09-18T01:08:51.911 INFO:teuthology.orchestra.run.smithi120.stdout:2407 write 0x84487c4 thru 0x8453eb2 (0xb6ef bytes) 2024-09-18T01:08:51.912 INFO:teuthology.orchestra.run.smithi120.stdout:2408 read 0x44467e2 thru 0x4447fb9 (0x17d8 bytes) 2024-09-18T01:08:51.913 INFO:teuthology.orchestra.run.smithi120.stdout:2410 punch from 0xdbffd9 to 0xdc8ae3, (0x8b0a bytes) 2024-09-18T01:08:51.913 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:51.913 INFO:teuthology.orchestra.run.smithi120.stdout:2414 write 0x86688ce thru 0x866fe90 (0x75c3 bytes) 2024-09-18T01:08:51.915 INFO:teuthology.orchestra.run.smithi120.stdout:2417 writesame 0xc645b38 thru 0xc64e6c6 data_size 0x1(0x8b8f bytes) 2024-09-18T01:08:51.939 INFO:teuthology.orchestra.run.smithi120.stdout:2418 write 0x69eb639 thru 0x69f7e81 (0xc849 bytes) 2024-09-18T01:08:51.942 INFO:teuthology.orchestra.run.smithi120.stdout:2419 punch from 0x34a5a3e to 0x34a6977, (0xf39 bytes) 2024-09-18T01:08:51.948 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:51.947+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:51.952 INFO:teuthology.orchestra.run.smithi120.stdout:2423 trunc from 0xe6d96b6 to 0x5bdccca 2024-09-18T01:08:51.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:51.965 INFO:teuthology.orchestra.run.smithi120.stdout:2425 writesame 0x4d8fd thru 0x5ac04 data_size 0xc(0xd308 bytes) 2024-09-18T01:08:51.969 INFO:teuthology.orchestra.run.smithi120.stdout:2426 punch from 0x116b7d4 to 0x117035e, (0x4b8a bytes) 2024-09-18T01:08:51.982 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:51.982 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:51.984 INFO:teuthology.orchestra.run.smithi120.stdout:2427 write 0xc0d7a6b thru 0xc0de12f (0x66c5 bytes) 2024-09-18T01:08:51.985 INFO:teuthology.orchestra.run.smithi120.stdout:2428 writesame 0x8d07e74 thru 0x8d11bb3 data_size 0x4a(0x9d40 bytes) 2024-09-18T01:08:51.987 INFO:teuthology.orchestra.run.smithi120.stdout:2429 punch from 0xb08d4eb to 0xb09c76c, (0xf281 bytes) 2024-09-18T01:08:51.993 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:51.995 INFO:teuthology.orchestra.run.smithi120.stdout:2430 writesame 0xd571c1f thru 0xd57d8ea data_size 0x56(0xbccc bytes) 2024-09-18T01:08:51.996 INFO:teuthology.orchestra.run.smithi120.stdout:2431 writesame 0x91ec3b7 thru 0x91f8129 data_size 0x1(0xbd73 bytes) 2024-09-18T01:08:52.026 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.026 INFO:teuthology.orchestra.run.smithi120.stdout:2433 punch from 0xb7b2d62 to 0xb7b8fc7, (0x6265 bytes) 2024-09-18T01:08:52.027 INFO:teuthology.orchestra.run.smithi120.stdout:2434 write 0x254e99 thru 0x262fc5 (0xe12d bytes) 2024-09-18T01:08:52.029 INFO:teuthology.orchestra.run.smithi120.stdout:2437 punch from 0x4a09fad to 0x4a0b5f0, (0x1643 bytes) 2024-09-18T01:08:52.029 INFO:teuthology.orchestra.run.smithi120.stdout:2439 read 0x8cc84ed thru 0x8ccd895 (0x53a9 bytes) 2024-09-18T01:08:52.029 INFO:teuthology.orchestra.run.smithi120.stdout:2441 punch from 0xd4d6dca to 0xd4dca07, (0x5c3d bytes) 2024-09-18T01:08:52.029 INFO:teuthology.orchestra.run.smithi120.stdout:2442 punch from 0xb268ae0 to 0xb269c62, (0x1182 bytes) 2024-09-18T01:08:52.029 INFO:teuthology.orchestra.run.smithi120.stdout:2443 writesame 0xda36c thru 0xe543e data_size 0x4f(0xb0d3 bytes) 2024-09-18T01:08:52.045 INFO:teuthology.orchestra.run.smithi120.stdout:2449 trunc from 0xd57d8eb to 0x17c46a5 2024-09-18T01:08:52.049 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:52.048+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:52.059 INFO:teuthology.orchestra.run.smithi120.stdout:2450 writesame 0x19ce48d thru 0x19dd9d0 data_size 0x16(0xf544 bytes) 2024-09-18T01:08:52.068 INFO:teuthology.orchestra.run.smithi120.stdout:2451 trunc from 0x19dd9d1 to 0x795dc95 2024-09-18T01:08:52.073 INFO:teuthology.orchestra.run.smithi120.stdout:2452 writesame 0x2d97d95 thru 0x2da0674 data_size 0x30(0x88e0 bytes) 2024-09-18T01:08:52.089 INFO:teuthology.orchestra.run.smithi120.stdout:2456 writesame 0xce61e15 thru 0xce6dab9 data_size 0x1(0xbca5 bytes) 2024-09-18T01:08:52.090 INFO:teuthology.orchestra.run.smithi120.stdout:2457 write 0xc86f1f3 thru 0xc877d59 (0x8b67 bytes) 2024-09-18T01:08:52.091 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.092 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.092 INFO:teuthology.orchestra.run.smithi120.stdout:2460 punch from 0xaf3ba92 to 0xaf4528c, (0x97fa bytes) 2024-09-18T01:08:52.095 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:52.096 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:52.109 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:52.109 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:52.113 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:52.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:52.120 INFO:teuthology.orchestra.run.smithi120.stdout:2461 writesame 0x38cbdd4 thru 0x38cddce data_size 0x1(0x1ffb bytes) 2024-09-18T01:08:52.124 INFO:teuthology.orchestra.run.smithi120.stdout:2462 read 0x5b1deed thru 0x5b20c8c (0x2da0 bytes) 2024-09-18T01:08:52.131 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:52.131 INFO:teuthology.orchestra.run.smithi120.stdout:2463 write 0xdf078ba thru 0xdf1652c (0xec73 bytes) 2024-09-18T01:08:52.133 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.133 INFO:teuthology.orchestra.run.smithi120.stdout:2465 write 0x694e9bd thru 0x694ebdb (0x21f bytes) 2024-09-18T01:08:52.136 INFO:teuthology.orchestra.run.smithi120.stdout:2467 write 0x16c12fd thru 0x16cd381 (0xc085 bytes) 2024-09-18T01:08:52.142 INFO:teuthology.orchestra.run.smithi120.stdout:2470 trunc from 0xdf1652d to 0xa4a0d77 2024-09-18T01:08:52.149 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:52.148+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:52.150 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:52.150 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:52.167 INFO:teuthology.orchestra.run.smithi120.stdout:2471 write 0xed88588 thru 0xed8d35c (0x4dd5 bytes) 2024-09-18T01:08:52.168 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:52.169 INFO:teuthology.orchestra.run.smithi120.stdout:2472 write 0x3dde82d thru 0x3de3c2d (0x5401 bytes) 2024-09-18T01:08:52.171 INFO:teuthology.orchestra.run.smithi120.stdout:2474 punch from 0x5da4d6e to 0x5db4ae5, (0xfd77 bytes) 2024-09-18T01:08:52.171 INFO:teuthology.orchestra.run.smithi120.stdout:2478 punch from 0x829c967 to 0x82a491a, (0x7fb3 bytes) 2024-09-18T01:08:52.171 INFO:teuthology.orchestra.run.smithi120.stdout:2480 read 0xb04333e thru 0xb04a5da (0x729d bytes) 2024-09-18T01:08:52.171 INFO:teuthology.orchestra.run.smithi120.stdout:2481 punch from 0xa56d6a3 to 0xa578156, (0xaab3 bytes) 2024-09-18T01:08:52.171 INFO:teuthology.orchestra.run.smithi120.stdout:2483 write 0xc846994 thru 0xc84b321 (0x498e bytes) 2024-09-18T01:08:52.172 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.172 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.173 INFO:teuthology.orchestra.run.smithi120.stdout:2489 read 0xd6c0b5a thru 0xd6cb358 (0xa7ff bytes) 2024-09-18T01:08:52.173 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.173 INFO:teuthology.orchestra.run.smithi120.stdout:2491 read 0x4a9e8bf thru 0x4aaa259 (0xb99b bytes) 2024-09-18T01:08:52.173 INFO:teuthology.orchestra.run.smithi120.stdout:2492 write 0x6ee78ce thru 0x6ef4ea5 (0xd5d8 bytes) 2024-09-18T01:08:52.180 INFO:teuthology.orchestra.run.smithi120.stdout:2493 trunc from 0xed8d35d to 0xb018320 2024-09-18T01:08:52.187 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:52.188 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:52.188 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.188 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.188 INFO:teuthology.orchestra.run.smithi120.stdout:2496 read 0x3113f4e thru 0x31225bf (0xe672 bytes) 2024-09-18T01:08:52.188 INFO:teuthology.orchestra.run.smithi120.stdout:2497 write 0x5e294d4 thru 0x5e2ab11 (0x163e bytes) 2024-09-18T01:08:52.189 INFO:teuthology.orchestra.run.smithi120.stdout:2498 punch from 0x3c425fe to 0x3c4a90d, (0x830f bytes) 2024-09-18T01:08:52.189 INFO:teuthology.orchestra.run.smithi120.stdout:2499 punch from 0x6408e43 to 0x6411788, (0x8945 bytes) 2024-09-18T01:08:52.189 INFO:teuthology.orchestra.run.smithi120.stdout:2503 read 0x11e4091 thru 0x11f39e8 (0xf958 bytes) 2024-09-18T01:08:52.189 INFO:teuthology.orchestra.run.smithi120.stdout:2504 write 0x154189b thru 0x154d595 (0xbcfb bytes) 2024-09-18T01:08:52.198 INFO:teuthology.orchestra.run.smithi120.stdout:2505 writesame 0xcd2e7eb thru 0xcd32603 data_size 0x7(0x3e19 bytes) 2024-09-18T01:08:52.199 INFO:teuthology.orchestra.run.smithi120.stdout:2506 read 0x860aa8a thru 0x86178c6 (0xce3d bytes) 2024-09-18T01:08:52.199 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.199 INFO:teuthology.orchestra.run.smithi120.stdout:2508 write 0xab77a69 thru 0xab77ffc (0x594 bytes) 2024-09-18T01:08:52.201 INFO:teuthology.orchestra.run.smithi120.stdout:2510 writesame 0xa428375 thru 0xa4333b0 data_size 0x2(0xb03c bytes) 2024-09-18T01:08:52.206 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:52.212 INFO:teuthology.orchestra.run.smithi120.stdout:2511 read 0xaba5ca9 thru 0xabb1faf (0xc307 bytes) 2024-09-18T01:08:52.213 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.213 INFO:teuthology.orchestra.run.smithi120.stdout:2513 write 0xa0e2cd8 thru 0xa0e4a3d (0x1d66 bytes) 2024-09-18T01:08:52.216 INFO:teuthology.orchestra.run.smithi120.stdout:2514 trunc from 0xcd32604 to 0xdfd9e64 2024-09-18T01:08:52.224 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:52.224 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:52.226 INFO:teuthology.orchestra.run.smithi120.stdout:2515 write 0xe000c83 thru 0xe006876 (0x5bf4 bytes) 2024-09-18T01:08:52.227 INFO:teuthology.orchestra.run.smithi120.stdout:2516 read 0x10a82bc thru 0x10b45e4 (0xc329 bytes) 2024-09-18T01:08:52.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:52.239 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.297212705999996, 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:08:52.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1074 3.0 e28/23 3:06c89d0b:::rbd_data.11769efc726f.000000000000000c:head v 28'5, mlcod=28'3)", 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.169284665999996, 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040496040000000001, 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067653+0000", 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067655+0000", 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7230000000000001e-06 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067802+0000", 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147093 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067821+0000", 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9383999999999999e-05 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067830+0000", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5189999999999992e-06 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067890+0000", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0362999999999998e-05 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.067915+0000", 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5003000000000001e-05 2024-09-18T01:08:52.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071653+0000", 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037380120000000002 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.071703+0000", 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9506999999999999e-05 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:1547 3.0 e28/23 3:01406944:::rbd_data.11769efc726f.0000000000000023:head v 28'10, mlcod=28'8)", 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.118910606, 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038816580000000001, 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118027+0000", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118029+0000", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118938+0000", 2024-09-18T01:08:52.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090941500000000001 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118957+0000", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.118966+0000", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5219999999999996e-06 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119032+0000", 2024-09-18T01:08:52.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5938000000000005e-05 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.119057+0000", 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5063999999999999e-05 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121856+0000", 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027984659999999999 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.121909+0000", 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2782999999999999e-05 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.029017390000007, 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:08:52.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.027421641999993, 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:08:52.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.026611023000001, 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.061877069000005, 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:08:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:08:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.059220366000005, 2024-09-18T01:08:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:08:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.476572822999998, 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:08:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:08:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.062259363999999, 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:08:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.560560342000002, 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:08:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:08:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.558983455000003, 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:08:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.529235864999997, 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:08:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.373856379000003, 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:08:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:08:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.370202949000003, 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:52190 3.4 e82/23 3:2587a2b7:::rbd_data.11762afc950e.0000000000000022:head v 82'135, mlcod=77'133)", 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.633986758999999, 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038334850000000002, 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602951+0000", 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:08:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602975+0000", 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3629999999999999e-05 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602984+0000", 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0820000000000005e-06 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.602992+0000", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6920000000000002e-06 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603100+0000", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107977 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.603124+0000", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4544000000000001e-05 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606735+0000", 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036103580000000001 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:29.606784+0000", 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9697000000000002e-05 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:08:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.654536274, 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:08:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 866 entries 100'101 (0'0) error 3:9153dd03:::rbd_data.117649d3c4d0.0000000000000093:head by client.4470.0:61161 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'99)", 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.808313101, 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048029769999999999, 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428633+0000", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7910000000000005e-06 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428653+0000", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109999999999999e-05 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428667+0000", 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4187e-05 2024-09-18T01:08:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428764+0000", 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.433427+0000", 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046639020000000002 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 869 entries 100'103 (0'0) error 3:94e57bb7:::rbd_data.117649d3c4d0.00000000000000b2:head by client.4470.0:61192 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7950263230000001, 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047167169999999996, 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441912+0000", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441920+0000", 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2260000000000005e-06 2024-09-18T01:08:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441942+0000", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1804e-05 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441957+0000", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4599e-05 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.442024+0000", 2024-09-18T01:08:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7860000000000004e-05 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.446628+0000", 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046035479999999998 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 871 entries 100'104 (0'0) error 3:90b555ad:::rbd_data.117649d3c4d0.00000000000000ba:head by client.4470.0:61200 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.792499404, 2024-09-18T01:08:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049806420000000004, 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444439+0000", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444445+0000", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444459+0000", 2024-09-18T01:08:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4241e-05 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444468+0000", 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5399999999999996e-06 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444541+0000", 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.449419+0000", 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048779060000000004 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.326 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:52.327 INFO:teuthology.orchestra.run.smithi120.stdout:2518 trunc from 0xe006877 to 0x5f8c44b 2024-09-18T01:08:52.328 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.328 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:52.328 INFO:teuthology.orchestra.run.smithi120.stdout:2521 write 0x4b8eeba thru 0x4b93988 (0x4acf bytes) 2024-09-18T01:08:52.329 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:52.329 INFO:teuthology.orchestra.run.smithi120.stdout:2522 write 0xa126e88 thru 0xa133760 (0xc8d9 bytes) 2024-09-18T01:08:52.330 INFO:teuthology.orchestra.run.smithi120.stdout:2524 read 0x2524269 thru 0x253338e (0xf126 bytes) 2024-09-18T01:08:52.330 INFO:teuthology.orchestra.run.smithi120.stdout:2525 read 0x972288a thru 0x972b8c3 (0x903a bytes) 2024-09-18T01:08:52.330 INFO:teuthology.orchestra.run.smithi120.stdout:2527 clone 25 order 18 su 2048 sc 11 2024-09-18T01:08:52.330 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:52.327+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:52.428 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:52.428+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:52.472 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:52.472 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:52.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:52.529 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:52.529+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:52.564 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:08:52.575 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:08:52.590 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:52.590 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:52.590 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:52.591 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:52.591 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:52.600 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:52.641 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:52.640+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:52.731 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:52.730+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:52.768 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:52.768 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:52.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38805 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2424832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193628854000004, 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244350999999999, 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38805 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.770 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:52.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569044+0000", 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569048+0000", 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7189999999999999e-06 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569057+0000", 2024-09-18T01:08:52.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4530000000000008e-06 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569060+0000", 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0170000000000001e-06 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611025+0000", 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041964317000000001 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611073+0000", 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8037000000000001e-05 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611288+0000", 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215596 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38806 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2588672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193609029000001, 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042508381999999997, 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38806 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569064+0000", 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4400000000000001e-07 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569065+0000", 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.206e-06 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569068+0000", 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2890000000000002e-06 2024-09-18T01:08:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569071+0000", 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611316+0000", 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042244918999999999 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611360+0000", 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4239000000000001e-05 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611572+0000", 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212186 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38807 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.781 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193599276, 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042802420000000001, 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38807 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569073+0000", 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569074+0000", 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569075+0000", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0219999999999999e-06 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569078+0000", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9749999999999999e-06 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569080+0000", 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:08:52.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611602+0000", 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042521681999999998 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611650+0000", 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8179000000000001e-05 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611876+0000", 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022609299999999999 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38808 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193589426999999, 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043110682999999997, 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38808 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569083+0000", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569084+0000", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569085+0000", 2024-09-18T01:08:52.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569088+0000", 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569090+0000", 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3120000000000001e-06 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611901+0000", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042810991999999999 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.611955+0000", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3983000000000001e-05 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612194+0000", 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239182 2024-09-18T01:08:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38809 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 475136~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193571102, 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043389637000000002, 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38809 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569102+0000", 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569122+0000", 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0254999999999998e-05 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569127+0000", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569130+0000", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.016e-06 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612224+0000", 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043094406000000002 2024-09-18T01:08:52.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612275+0000", 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1054000000000003e-05 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612491+0000", 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215987 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38810 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193538255999997, 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043652712000000003, 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38810 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569134+0000", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569135+0000", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569136+0000", 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.083e-06 2024-09-18T01:08:52.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569141+0000", 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569143+0000", 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3369999999999998e-06 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612521+0000", 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043377538 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612566+0000", 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5485000000000002e-05 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612787+0000", 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221158 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38811 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193526974999997, 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043930856999999997, 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38811 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569146+0000", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569147+0000", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0449999999999999e-06 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569150+0000", 2024-09-18T01:08:52.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9629999999999998e-06 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569152+0000", 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612830+0000", 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043677597999999998 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.612880+0000", 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0219999999999997e-05 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613077+0000", 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019646600000000001 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38812 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193517765999999, 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044183260000000002, 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38812 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569155+0000", 2024-09-18T01:08:52.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569156+0000", 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999991e-07 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569160+0000", 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3759999999999999e-06 2024-09-18T01:08:52.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569162+0000", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613107+0000", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043945166000000001 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613153+0000", 2024-09-18T01:08:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613338+0000", 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018564800000000001 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38813 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1130496~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:52.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193508297999998, 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044519186000000002, 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38813 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569164+0000", 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569165+0000", 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569166+0000", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2399999999999996e-07 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569168+0000", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9229999999999998e-06 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569178+0000", 2024-09-18T01:08:52.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6479999999999998e-06 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613370+0000", 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044191800000000003 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613416+0000", 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6431999999999999e-05 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613684+0000", 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026725300000000001 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38814 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193491467000001, 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044848796000000003, 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38814 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569181+0000", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569183+0000", 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.096e-06 2024-09-18T01:08:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569187+0000", 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569191+0000", 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1930000000000002e-06 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613735+0000", 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044543886999999997 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.613784+0000", 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9858999999999998e-05 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614030+0000", 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024569700000000002 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38815 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1458176~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193479355999997, 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045101439, 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.820 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38815 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569193+0000", 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569194+0000", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569195+0000", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0389999999999999e-06 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569198+0000", 2024-09-18T01:08:52.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9210000000000001e-06 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569200+0000", 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.367e-06 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614061+0000", 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.044861375000000002 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614104+0000", 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3142000000000001e-05 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614295+0000", 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190383 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38816 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1622016~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.824 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193470245999997, 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045464629999999999, 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38816 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569202+0000", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569203+0000", 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569206+0000", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.754e-06 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569211+0000", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.109e-06 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569213+0000", 2024-09-18T01:08:52.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.379e-06 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614331+0000", 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045118395999999998 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614380+0000", 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614667+0000", 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028724000000000002 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38817 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1785856~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193456853000001, 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045799302, 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38817 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569216+0000", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569217+0000", 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.035e-06 2024-09-18T01:08:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569221+0000", 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.839e-06 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569223+0000", 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3300000000000001e-06 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614711+0000", 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045487811000000003 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.614772+0000", 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0729000000000001e-05 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615015+0000", 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024334399999999999 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38818 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193446803000001, 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046120525000000002, 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38818 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569226+0000", 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569230+0000", 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0260000000000001e-06 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569234+0000", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0890000000000002e-06 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569237+0000", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3750000000000001e-06 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615046+0000", 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045809435000000003 2024-09-18T01:08:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615096+0000", 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9654999999999998e-05 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615346+0000", 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025073700000000001 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38819 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2113536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193431015999998, 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046434472999999997, 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38819 2024-09-18T01:08:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569242+0000", 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569243+0000", 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.291e-06 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569247+0000", 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.031e-06 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569250+0000", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615395+0000", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046145046000000002 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615437+0000", 2024-09-18T01:08:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2407999999999998e-05 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615676+0000", 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239137 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38820 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2277376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193420117999999, 2024-09-18T01:08:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046745223000000002, 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38820 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569253+0000", 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569254+0000", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7999999999999993e-07 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569258+0000", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6489999999999999e-06 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569261+0000", 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.322e-06 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615704+0000", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046442986999999998 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615764+0000", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9871000000000003e-05 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.615998+0000", 2024-09-18T01:08:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023420099999999999 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38821 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2441216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193409266000003, 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047015754999999999, 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38821 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569263+0000", 2024-09-18T01:08:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569264+0000", 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569265+0000", 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8599999999999996e-07 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569269+0000", 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8800000000000001e-06 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569271+0000", 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616017+0000", 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.046746593000000003 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616061+0000", 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3894999999999999e-05 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616279+0000", 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000217875 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38822 3.18 3:1dee1f34:::rbd_data.1176f27201d2.0000000000000000:19 [read 2605056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.193399194000001, 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047344570000000002, 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38822 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569274+0000", 2024-09-18T01:08:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1e-07 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569275+0000", 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.026e-06 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569278+0000", 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999999e-06 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.569284+0000", 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.071e-06 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616318+0000", 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.047033859999999997 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616374+0000", 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5636000000000001e-05 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616618+0000", 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024412699999999999 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38823 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.189422620999999, 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.045935959999999998, 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38823 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573250+0000", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573251+0000", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573255+0000", 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2520000000000001e-06 2024-09-18T01:08:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573266+0000", 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0958000000000001e-05 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.573271+0000", 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1229999999999999e-06 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616299+0000", 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.043027351999999998 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616351+0000", 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2284000000000003e-05 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.616681+0000", 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033019400000000002 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619078+0000", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023970749999999998 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619093+0000", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4338999999999999e-05 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619143+0000", 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0346000000000002e-05 2024-09-18T01:08:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.619186+0000", 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3226999999999998e-05 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:38824 3.8 3:12a65bb9:::rbd_object_map.11762bc2f01c.000000000000001a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.184083272000002, 2024-09-18T01:08:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.042771339999999998, 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 38824 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578589+0000", 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578590+0000", 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578594+0000", 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0609999999999997e-06 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578605+0000", 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0477e-05 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.578611+0000", 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9540000000000001e-06 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617084+0000", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.038473680000000003 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617135+0000", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0139000000000002e-05 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.617316+0000", 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181082 2024-09-18T01:08:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621302+0000", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039860210000000002 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621309+0000", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6490000000000008e-06 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621336+0000", 2024-09-18T01:08:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6828000000000001e-05 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.621361+0000", 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4621999999999999e-05 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:52.869 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:52.871 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:52.871 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:52.870+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:52.973 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:52.972+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:52.975 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:52.975 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:52.975 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:52.975 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:52.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:53.074 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:53.073+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:53.086 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:53.087 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:53.087 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:53.087 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:53.087 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:53.095 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:53.175 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:53.174+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8309 3.1b e34/23 3:dd5c2af2:::rbd_data.11762d3aac1.0000000000000001:head v 34'130, mlcod=32'128)", 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.646671002000005, 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015697204999999999, 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591460+0000", 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.591465+0000", 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3869999999999997e-06 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599747+0000", 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082827379999999996 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599763+0000", 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5262000000000001e-05 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599770+0000", 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.917e-06 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599819+0000", 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9493000000000002e-05 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599837+0000", 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7646e-05 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607116+0000", 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072794180000000002 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607158+0000", 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1344000000000003e-05 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8347 3.1 e34/23 3:822d540a:::rbd_data.11762d3aac1.0000000000000002:head v 34'13, mlcod=31'11)", 2024-09-18T01:08:53.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.638339329999994, 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012816164999999999, 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599792+0000", 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.599793+0000", 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999999e-07 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603065+0000", 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032717390000000001 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603081+0000", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6243e-05 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603089+0000", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9049999999999997e-06 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603184+0000", 2024-09-18T01:08:53.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5278999999999994e-05 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603214+0000", 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0159999999999999e-05 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612572+0000", 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093581759999999993 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.612608+0000", 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5880999999999997e-05 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8372 3.1 e34/23 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head v 34'14, mlcod=31'11)", 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.634998476000007, 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012783045999999999, 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603133+0000", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.603134+0000", 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8999999999999996e-07 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606684+0000", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035506579999999999 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606699+0000", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4934e-05 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.606707+0000", 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0520000000000002e-06 2024-09-18T01:08:53.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607674+0000", 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096683200000000004 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.607698+0000", 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4213e-05 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615852+0000", 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0081536430000000003 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.615916+0000", 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4023999999999996e-05 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8786 3.1 e34/23 3:85a569e1:::rbd_data.11762d3aac1.0000000000000008:head v 34'15, mlcod=31'12)", 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.607543647, 2024-09-18T01:08:53.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022484270000000001, 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630588+0000", 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630589+0000", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0380000000000001e-06 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639089+0000", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0085006189999999992 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639102+0000", 2024-09-18T01:08:53.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2614e-05 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639112+0000", 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9559999999999992e-06 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639241+0000", 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129479 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.639265+0000", 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3359000000000001e-05 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653009+0000", 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013743907 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653072+0000", 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3298000000000003e-05 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.573578330000004, 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:53.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:53.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12800 3.1a 3:5a55d197:::rbd_data.1176ec0d676b.000000000000000f:head [sparse-read 0~1572864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.492610744000004, 2024-09-18T01:08:53.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013316737, 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12800 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:53.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745521+0000", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745524+0000", 2024-09-18T01:08:53.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4389999999999999e-06 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745530+0000", 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745534+0000", 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9250000000000003e-06 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753853+0000", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0083188610000000003 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.753945+0000", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1949999999999999e-05 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.758837+0000", 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048920880000000002 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:12801 3.9 3:977c0ffb:::rbd_data.1176ec0d676b.0000000000000010:head [sparse-read 0~1672895] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.492588905999995, 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012390297999999999, 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.268 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 12801 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745542+0000", 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745543+0000", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745545+0000", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1959999999999998e-06 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745552+0000", 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1199999999999996e-06 2024-09-18T01:08:53.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.745558+0000", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6620000000000002e-06 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.751981+0000", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064229409999999997 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.752075+0000", 2024-09-18T01:08:53.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4624999999999996e-05 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:44.757933+0000", 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058573330000000002 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:17726 3.f e46/23 3:f1141c5e:::rbd_data.11762d95e74a.0000000000000001:head v 46'27, mlcod=44'25)", 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:53.272 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.602243606000002, 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011268102, 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.635888+0000", 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636279+0000", 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000390813 2024-09-18T01:08:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636289+0000", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4199999999999996e-06 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636295+0000", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000002e-06 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636339+0000", 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4613999999999997e-05 2024-09-18T01:08:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.636359+0000", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9497999999999999e-05 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647101+0000", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010742289 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647156+0000", 2024-09-18T01:08:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4827999999999998e-05 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17764 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.600700740999997, 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098372939999999999, 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17764 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637431+0000", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637433+0000", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.187e-06 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637439+0000", 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:08:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637444+0000", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3380000000000004e-06 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646651+0000", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0092067580000000006 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.646713+0000", 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1963000000000003e-05 2024-09-18T01:08:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647268+0000", 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000554935 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17765 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 294912~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.600679583999998, 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012716865000000001, 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17765 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637452+0000", 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637454+0000", 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637459+0000", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0440000000000003e-06 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637464+0000", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-06 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647365+0000", 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099005650000000001 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.647459+0000", 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4708000000000002e-05 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650169+0000", 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002709267 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:17766 3.14 3:2b90fde6:::rbd_data.1176492bc9ca.000000000000000b:f [read 393216~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:08:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 60.600660769999998, 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01329845, 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 17766 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637471+0000", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637473+0000", 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2129999999999998e-06 2024-09-18T01:08:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637480+0000", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.637485+0000", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8210000000000001e-06 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650199+0000", 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012713482 2024-09-18T01:08:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650278+0000", 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9519999999999998e-05 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:52.650769+0000", 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049094199999999996 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27797 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.147142594000002, 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096847760000000008, 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27797 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090989+0000", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4900000000000002e-07 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090990+0000", 2024-09-18T01:08:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3820000000000001e-06 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.090994+0000", 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091000+0000", 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094887+0000", 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003887471 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.094993+0000", 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105955 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095247+0000", 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254279 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100245+0000", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049975790000000003 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100608+0000", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000363459 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100643+0000", 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5247999999999999e-05 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100674+0000", 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0059000000000001e-05 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:27798 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.147128674000001, 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098152059999999999, 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 27798 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091003+0000", 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4499999999999998e-07 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091004+0000", 2024-09-18T01:08:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.423e-06 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091008+0000", 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7009999999999998e-06 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.091011+0000", 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5390000000000002e-06 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095569+0000", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045587589999999999 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095663+0000", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3787000000000002e-05 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.095894+0000", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023077300000000001 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100250+0000", 2024-09-18T01:08:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004355678 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100718+0000", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000468352 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100772+0000", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3918000000000001e-05 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.100818+0000", 2024-09-18T01:08:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6031e-05 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28354 3.14 3:2c6af8ed:::rbd_object_map.1176e641238f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 55.670582609999997, 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096460690000000002, 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28354 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567549+0000", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567550+0000", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567554+0000", 2024-09-18T01:08:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1099999999999999e-06 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.567556+0000", 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569616+0000", 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020597950000000001 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569663+0000", 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6828e-05 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.569889+0000", 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226324 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.574073+0000", 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041843389999999996 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577132+0000", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030587639999999998 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577166+0000", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3420999999999997e-05 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.577195+0000", 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9241999999999999e-05 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 55.515390750999998, 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28530 3.a 3:517d6535:::rbd_data.1176e641238f.0000000000000018:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.578384751999998, 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096917310000000003, 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28530 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659747+0000", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659750+0000", 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659757+0000", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4619999999999999e-06 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.659761+0000", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6090000000000001e-06 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668040+0000", 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0082796199999999997 2024-09-18T01:08:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668163+0000", 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122564 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.668373+0000", 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021029600000000001 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669298+0000", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00092485600000000005 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669378+0000", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9546e-05 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669410+0000", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2212000000000003e-05 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.669438+0000", 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8387000000000001e-05 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:39493 3.1a 3:5ccc4d1f:::rbd_data.11762bc2f01c.0000000000000026:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.514434239000003, 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098167740000000003, 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 39493 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723697+0000", 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723698+0000", 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.079e-06 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723945+0000", 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000246333 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723956+0000", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1781e-05 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.723964+0000", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3719999999999999e-06 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724038+0000", 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4203999999999996e-05 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724541+0000", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050327399999999995 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.724703+0000", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161948 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.726843+0000", 2024-09-18T01:08:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021401990000000002 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733394+0000", 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00655034 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733452+0000", 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8579999999999998e-05 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.733514+0000", 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1663999999999998e-05 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:40715 3.3 3:c691eadd:::rbd_data.11762bc2f01c.0000000000000063:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.376690637000003, 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010922535000000001, 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 40715 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861441+0000", 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861443+0000", 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9120000000000001e-06 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861668+0000", 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022575399999999999 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861684+0000", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5639000000000001e-05 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861691+0000", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3599999999999998e-06 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861744+0000", 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2636000000000002e-05 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861844+0000", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010024199999999999 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.861962+0000", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117237 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.863824+0000", 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018622039999999999 2024-09-18T01:08:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872247+0000", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084235569999999999 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872320+0000", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2646999999999997e-05 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:10.872363+0000", 2024-09-18T01:08:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3346999999999997e-05 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:58523 3.1a 3:5a42e52b:::rbd_data.11768408e4d7.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7759659079999999, 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.014664071000000001, 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 58523 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462165+0000", 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462166+0000", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8700000000000005e-07 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462171+0000", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2939999999999999e-06 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462181+0000", 2024-09-18T01:08:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0908e-05 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462187+0000", 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462245+0000", 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8279999999999998e-05 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.462294+0000", 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8729000000000001e-05 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.465104+0000", 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028099100000000001 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476746+0000", 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011641979 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476760+0000", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4087000000000001e-05 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476793+0000", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3769999999999997e-05 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:44.476830+0000", 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6171000000000003e-05 2024-09-18T01:08:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:61531 3.1a 3:5f2ef990:::rbd_data.117649d3c4d0.00000000000000d8:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.3958677379999997, 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016342980999999999, 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 61531 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842264+0000", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842265+0000", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7599999999999996e-07 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842269+0000", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9150000000000004e-06 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842281+0000", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1270000000000001e-05 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842286+0000", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842325+0000", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9431000000000002e-05 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.842419+0000", 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3338000000000007e-05 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.845634+0000", 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032156099999999998 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858476+0000", 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012841491 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858494+0000", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8533e-05 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858555+0000", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0372999999999999e-05 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:48.858607+0000", 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1861999999999999e-05 2024-09-18T01:08:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.343 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:53.344 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:53.345 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:53.343+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:53.445 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:53.444+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:53.488 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:53.488 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:53.488 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:53.488 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:53.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:53.546 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:53.545+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:53.569 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:08:53.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:08:53.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:08:53.620 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:53.620 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:53.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:53.675 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:53.674+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:53.747 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:53.746+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:53.822 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796172627999994, 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796130477000005, 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796097048999997, 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796059549000006, 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796037971999993, 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.796018832000001, 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795998440999995, 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795969041000006, 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795940251000005, 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795919556000001, 2024-09-18T01:08:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:53.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795900071999995, 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:53.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795866161000006, 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.877 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:53.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:53.881 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.795830162000001, 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:53.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.711382189999995, 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:53.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:53.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.711371980999999, 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:53.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:53.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.210970755000005, 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:53.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.195484586000006, 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:53.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:53.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:53.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:53.906 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.187593866, 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.161524862999997, 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:53.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:53.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.156356071000005, 2024-09-18T01:08:53.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:53.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:53.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:53.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:53.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:53.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:53.922 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:53.923 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:53.924 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:53.922+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:54.036 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:08:54.035+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:08:54.123 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:54.123 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:54.123 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:54.123 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:54.124 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:54.124+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:54.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:54.226 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:54.225+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:54.319 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:54.319 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:54.319 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:54.319 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:54.319 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:54.327 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:54.325+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:54.327 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:54.328 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:54.427 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:54.426+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:54.449 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:54.478 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:54.478 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:54.486 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.912306516000001, 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:08:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.860315372999999, 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.860238577000001, 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:08:54.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.855733424, 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:08:54.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.855477627999999, 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:08:54.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.851452299999998, 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:08:54.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:08:54.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:08:54.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.851311150000001, 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:54.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:08:54.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.851078246, 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:08:54.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:08:54.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.848459183999999, 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:08:54.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.826120150000001, 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:08:54.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:54.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87560487600000003, 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:54.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87558313499999996, 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:08:54.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87556272700000004, 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:08:54.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:08:54.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:08:54.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87554306800000004, 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:08:54.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87550973300000001, 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:08:54.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87547520099999998, 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:08:54.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87543031100000002, 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87540393599999999, 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:08:54.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:54.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87538478500000005, 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:54.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:08:54.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:08:54.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:54.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.87536621699999995, 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:54.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:08:54.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:08:54.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:54.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:54.591 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:54.592 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:54.593 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:54.594 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:54.594 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:54.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:54.595 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:54.591+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:54.596 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:54.596 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:54.597 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:08:54.597 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:08:54.598 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:08:54.601 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:54.602 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:54.628 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:54.628 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:54.650 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:54.650 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:54.705 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:54.704+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:54.790 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:54.790 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:54.794 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:54.793+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:54.799 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:54.895 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:54.894+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:54.954 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:54.954 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:54.967 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:54.996 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:54.995+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:55.097 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:55.096+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:55.126 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:55.127 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:55.140 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:55.198 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:55.197+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:55.299 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:55.298+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:55.314 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:55.314 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:55.329 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:55.401 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:55.400+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:55.497 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:55.497 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:55.501 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:55.500+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:55.512 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:55.580 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:08:55.603 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:55.601+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:55.604 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:08:55.704 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:55.703+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:55.739 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:55.739 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:55.754 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:55.804 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:55.803+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:55.905 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:55.904+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:55.976 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:55.976 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:55.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:08:56.005 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:56.004+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:56.106 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:56.105+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:56.151 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:56.151 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:56.163 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:08:56.207 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:56.206+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:56.291 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:56.291 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:56.301 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:08:56.307 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:56.306+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:56.408 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:56.407+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:56.464 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:56.464 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:56.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:08:56.515 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:56.514+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:56.591 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:08:56.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:08:56.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:08:56.610 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:56.608+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:56.646 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:56.646 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:56.658 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:08:56.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:08:56.744 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:56.743+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:56.810 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:56.809+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:56.842 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:56.842 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:56.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:08:56.859 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:56.878 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:56.907 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:56.908 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:56.911 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:56.910+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:56.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:56.926 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:56.942 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:56.943 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:56.959 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:56.959 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:56.965 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:56.965 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:56.965 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:08:56.965 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:56.974 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:08:56.977 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:56.977 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:56.994 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:56.994 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:57.013 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:57.012+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:57.080 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:57.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:08:57.080 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:08:57.080 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:08:57.080 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:57.089 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:08:57.133 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:57.132+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:57.192 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:08:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:08:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:08:57.192 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5622278270000001, 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:08:57.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:08:57.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.197 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562202842, 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:08:57.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562183063, 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:08:57.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:08:57.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5621566609999999, 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.210 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5621351649999999, 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:08:57.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5621158689999999, 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.215 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:08:57.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:08:57.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562098035, 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:08:57.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:08:57.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562071972, 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:08:57.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:08:57.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562052617, 2024-09-18T01:08:57.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:08:57.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.562022373, 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:08:57.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5620037780000002, 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:08:57.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:08:57.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5619852189999999, 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:08:57.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:08:57.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.561967165, 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5619494060000001, 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.255 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.561923137, 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5619045169999999, 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:08:57.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:08:57.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5618862670000002, 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:08:57.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5613348920000001, 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:08:57.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5597359060000002, 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:08:57.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5559699189999998, 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:08:57.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:08:57.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:08:57.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:08:57.284 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:08:57.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:08:57.286 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:57.284+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:57.386 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:57.385+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:57.401 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:57.401 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:57.410 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:08:57.486 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:57.486+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:57.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:08:57.587 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:57.586+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:57.597 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:08:57.600 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:08:57.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:08:57.632 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.965487843000005, 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.907300264, 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:08:57.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:57.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:08:57.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:57.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:08:57.644 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.767804661, 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.648 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7677821419999997, 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.649 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.767760574, 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:08:57.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:08:57.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7677398050000002, 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:08:57.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:08:57.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.76772061, 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:08:57.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.767701824, 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:08:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:08:57.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:08:57.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7676825149999997, 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.767660126, 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.677 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:08:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.681 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7676388059999999, 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:08:57.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:08:57.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.767615213, 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:08:57.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:08:57.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:08:57.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7675952969999997, 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:08:57.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7675768659999997, 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:08:57.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7675526019999999, 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.700 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:08:57.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:08:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:08:57.704 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7675282279999998, 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:08:57.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7675070879999999, 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:08:57.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.713 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7674870060000001, 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:08:57.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:08:57.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7674668599999999, 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.718 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:08:57.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.722 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7674467969999998, 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:08:57.723 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:08:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:08:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:08:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:57.728 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:57.730 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-18T01:08:57.730 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:08:57.731 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:57.728+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:57.830 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:57.829+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:57.881 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:57.881 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:57.889 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:08:57.931 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:57.930+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:57.993 INFO:teuthology.orchestra.run.smithi120.stdout:2528 read 0x4f97d29 thru 0x4fa242a (0xa702 bytes) 2024-09-18T01:08:57.994 INFO:teuthology.orchestra.run.smithi120.stdout:2529 read 0x42158ee thru 0x421d783 (0x7e96 bytes) 2024-09-18T01:08:58.012 INFO:teuthology.orchestra.run.smithi120.stdout:2530 writesame 0xc8c25b1 thru 0xc8c90cc data_size 0x1e(0x6b1c bytes) 2024-09-18T01:08:58.012 INFO:teuthology.orchestra.run.smithi120.stdout:2531 read 0xa9fdf26 thru 0xaa0cc25 (0xed00 bytes) 2024-09-18T01:08:58.012 INFO:teuthology.orchestra.run.smithi120.stdout:2532 read 0xc018829 thru 0xc022d1d (0xa4f5 bytes) 2024-09-18T01:08:58.012 INFO:teuthology.orchestra.run.smithi120.stdout:2533 write 0x4b7c115 thru 0x4b81f19 (0x5e05 bytes) 2024-09-18T01:08:58.017 INFO:teuthology.orchestra.run.smithi120.stdout:2535 write 0x780e64 thru 0x78f849 (0xe9e6 bytes) 2024-09-18T01:08:58.027 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:58.027 INFO:teuthology.orchestra.run.smithi120.stdout:2537 punch from 0xae015ef to 0xae0b970, (0xa381 bytes) 2024-09-18T01:08:58.031 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:58.030+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:58.032 INFO:teuthology.orchestra.run.smithi120.stdout:2538 trunc from 0xc8c90cd to 0x892fb41 2024-09-18T01:08:58.039 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.039 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:58.039 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:58.039 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:58.039 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:08:58.071 INFO:teuthology.orchestra.run.smithi120.stdout:2539 punch from 0x58cabe2 to 0x58d005a, (0x5478 bytes) 2024-09-18T01:08:58.071 INFO:teuthology.orchestra.run.smithi120.stdout:2541 read 0x6d03b61 thru 0x6d0cd4b (0x91eb bytes) 2024-09-18T01:08:58.072 INFO:teuthology.orchestra.run.smithi120.stdout:2542 read 0x4668058 thru 0x4676686 (0xe62f bytes) 2024-09-18T01:08:58.072 INFO:teuthology.orchestra.run.smithi120.stdout:2543 writesame 0x3a18119 thru 0x3a1c434 data_size 0xa(0x431c bytes) 2024-09-18T01:08:58.080 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:58.090 INFO:teuthology.orchestra.run.smithi120.stdout:2545 trunc from 0x892fb41 to 0x7735 2024-09-18T01:08:58.135 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:08:58.134+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203818949999999, 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:08:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:08:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203776798999996, 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:08:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203743371000002, 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203705870999997, 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:08:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203684293999999, 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:08:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:08:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203665154000007, 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:08:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203644763, 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:08:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203615362999997, 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:08:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:08:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203586572999995, 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:08:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203565878000006, 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:08:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203546394, 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:08:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203512482999997, 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.203476484000007, 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:08:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:08:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.119028512, 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 83.119018303000004, 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:08:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:08:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.618617076999996, 2024-09-18T01:08:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:08:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.603130907999997, 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:08:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:08:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:08:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.595240188000005, 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:08:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.569171185000002, 2024-09-18T01:08:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:08:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:08:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:08:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.564002392999996, 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:08:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.328 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:2547 read 0x713f thru 0x7734 (0x5f6 bytes) 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:2548 punch from 0x6dfc to 0x7735, (0x939 bytes) 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:2550 read 0x2031 thru 0x7734 (0x5704 bytes) 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:2551 read 0x989 thru 0x5f62 (0x55da bytes) 2024-09-18T01:08:58.330 INFO:teuthology.orchestra.run.smithi120.stdout:2552 clone 26 order 22 su 65536 sc 8 2024-09-18T01:08:58.330 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:58.328+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:58.331 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:08:58.430 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:58.429+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:58.473 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.474 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:58.474 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:58.474 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.482 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:08:58.530 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:58.529+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.585 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:58.586 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:58.586 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:08:58.598 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:08:58.614 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:08:58.614 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:08:58.655 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:58.654+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:58.732 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:58.731+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:58.739 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.166817656999999, 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:58.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:08:58.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:08:58.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:58.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.114826514000001, 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:08:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.114749717999999, 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:08:58.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.757 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.110244564999999, 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:08:58.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.109988769000001, 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:08:58.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.105963441, 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:08:58.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.105822290999999, 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:08:58.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:08:58.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.105589386999998, 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:08:58.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:08:58.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.102970325000001, 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:08:58.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:08:58.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.080631291, 2024-09-18T01:08:58.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:08:58.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:08:58.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1301160169999998, 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.796 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1300942760000003, 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:08:58.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1300738680000002, 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:08:58.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:08:58.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1300542089999999, 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:08:58.805 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:08:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1300208740000004, 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.810 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:08:58.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.814 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1299863419999996, 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:08:58.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:08:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:08:58.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.819 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1299414519999997, 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:08:58.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:08:58.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:58.824 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1299150769999997, 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:58.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1298959259999997, 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:08:58.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1298773579999999, 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:08:58.833 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:08:58.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:58.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:58.838 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.839 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:08:58.839 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:58.838+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:58.930 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:08:58.940 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:08:58.939+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:08:58.941 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-18T01:08:58.739034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1075072921,"num_objects":1275,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1275,"num_whiteouts":0,"num_read":108553,"num_read_kb":766819,"num_write":14076,"num_write_kb":507874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9581,"ondisk_log_size":9581,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972196,"kb_used_data":643572,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754002524,"statfs":{"total":773094113280,"available":772098584576,"internally_reserved":0,"allocated":659017728,"data_stored":650080758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,16,5,27,107,93],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":128899442,"num_objects":470,"num_object_clones":0,"num_object_copies":940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":47938,"num_read_kb":86013,"num_write":1777,"num_write_kb":79950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000679"},"pg_stats":[{"pgid":"3.1f","version":"109'240","reported_seq":1887,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.602923+0000","last_change":"2024-09-18T01:08:57.603130+0000","last_active":"2024-09-18T01:08:57.602923+0000","last_peered":"2024-09-18T01:08:57.602923+0000","last_clean":"2024-09-18T01:08:57.602923+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:08:57.602923+0000","last_undegraded":"2024-09-18T01:08:57.602923+0000","last_fullsized":"2024-09-18T01:08:57.602923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:34.552259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042484100000000002,"stat_sum":{"num_bytes":29947159,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1896,"num_read_kb":20363,"num_write":264,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1e","version":"106'144","reported_seq":1000,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577315+0000","last_change":"2024-09-18T01:08:51.566309+0000","last_active":"2024-09-18T01:08:54.577315+0000","last_peered":"2024-09-18T01:08:54.577315+0000","last_clean":"2024-09-18T01:08:54.577315+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:08:54.577315+0000","last_undegraded":"2024-09-18T01:08:54.577315+0000","last_fullsized":"2024-09-18T01:08:54.577315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:46.316335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062472000000000005,"stat_sum":{"num_bytes":12934403,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":773,"num_read_kb":16830,"num_write":221,"num_write_kb":10618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"110'175","reported_seq":1537,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.609311+0000","last_change":"2024-09-18T01:08:57.609311+0000","last_active":"2024-09-18T01:08:57.609311+0000","last_peered":"2024-09-18T01:08:57.609311+0000","last_clean":"2024-09-18T01:08:57.609311+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:08:57.609311+0000","last_undegraded":"2024-09-18T01:08:57.609311+0000","last_fullsized":"2024-09-18T01:08:57.609311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:47.480704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000911122,"stat_sum":{"num_bytes":33777976,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1013,"num_read_kb":27852,"num_write":252,"num_write_kb":31132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1c","version":"109'311","reported_seq":1579,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608467+0000","last_change":"2024-09-18T01:08:57.608635+0000","last_active":"2024-09-18T01:08:57.608467+0000","last_peered":"2024-09-18T01:08:57.608467+0000","last_clean":"2024-09-18T01:08:57.608467+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:08:57.608467+0000","last_undegraded":"2024-09-18T01:08:57.608467+0000","last_fullsized":"2024-09-18T01:08:57.608467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:32.662596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061793899999999997,"stat_sum":{"num_bytes":30508630,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":1825,"num_read_kb":25229,"num_write":447,"num_write_kb":26223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1b","version":"109'901","reported_seq":3374,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603581+0000","last_change":"2024-09-18T01:08:57.603782+0000","last_active":"2024-09-18T01:08:57.603581+0000","last_peered":"2024-09-18T01:08:57.603581+0000","last_clean":"2024-09-18T01:08:57.603581+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:08:57.603581+0000","last_undegraded":"2024-09-18T01:08:57.603581+0000","last_fullsized":"2024-09-18T01:08:57.603581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:54.201618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007729,"stat_sum":{"num_bytes":28425303,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":6509,"num_read_kb":29048,"num_write":1351,"num_write_kb":20735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1a","version":"107'162","reported_seq":1820,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.589912+0000","last_change":"2024-09-18T01:08:51.567312+0000","last_active":"2024-09-18T01:08:55.589912+0000","last_peered":"2024-09-18T01:08:55.589912+0000","last_clean":"2024-09-18T01:08:55.589912+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:08:55.589912+0000","last_undegraded":"2024-09-18T01:08:55.589912+0000","last_fullsized":"2024-09-18T01:08:55.589912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:19.119192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643299999999999,"stat_sum":{"num_bytes":18574012,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1618,"num_read_kb":21557,"num_write":218,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.19","version":"109'159","reported_seq":1599,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.715628+0000","last_change":"2024-09-18T01:08:57.600053+0000","last_active":"2024-09-18T01:08:57.715628+0000","last_peered":"2024-09-18T01:08:57.715628+0000","last_clean":"2024-09-18T01:08:57.715628+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:08:57.715628+0000","last_undegraded":"2024-09-18T01:08:57.715628+0000","last_fullsized":"2024-09-18T01:08:57.715628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:37.265835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043584400000000001,"stat_sum":{"num_bytes":35778665,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1149,"num_read_kb":19160,"num_write":189,"num_write_kb":14501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.18","version":"106'183","reported_seq":4721,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.750358+0000","last_change":"2024-09-18T01:08:51.566328+0000","last_active":"2024-09-18T01:08:54.750358+0000","last_peered":"2024-09-18T01:08:54.750358+0000","last_clean":"2024-09-18T01:08:54.750358+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:08:54.750358+0000","last_undegraded":"2024-09-18T01:08:54.750358+0000","last_fullsized":"2024-09-18T01:08:54.750358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:09.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012726899999999999,"stat_sum":{"num_bytes":42968641,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4268,"num_read_kb":35281,"num_write":220,"num_write_kb":16625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"109'315","reported_seq":1382,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603638+0000","last_change":"2024-09-18T01:08:57.603853+0000","last_active":"2024-09-18T01:08:57.603638+0000","last_peered":"2024-09-18T01:08:57.603638+0000","last_clean":"2024-09-18T01:08:57.603638+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:08:57.603638+0000","last_undegraded":"2024-09-18T01:08:57.603638+0000","last_fullsized":"2024-09-18T01:08:57.603638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:00:41.195936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098992299999999997,"stat_sum":{"num_bytes":34021606,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1551,"num_read_kb":21053,"num_write":482,"num_write_kb":15397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.16","version":"108'156","reported_seq":1256,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608210+0000","last_change":"2024-09-18T01:08:57.608418+0000","last_active":"2024-09-18T01:08:57.608210+0000","last_peered":"2024-09-18T01:08:57.608210+0000","last_clean":"2024-09-18T01:08:57.608210+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:08:57.608210+0000","last_undegraded":"2024-09-18T01:08:57.608210+0000","last_fullsized":"2024-09-18T01:08:57.608210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:57:19.715620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395764,"stat_sum":{"num_bytes":30840218,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":861,"num_read_kb":14934,"num_write":191,"num_write_kb":11808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.15","version":"109'249","reported_seq":5740,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.622154+0000","last_change":"2024-09-18T01:08:57.622360+0000","last_active":"2024-09-18T01:08:57.622154+0000","last_peered":"2024-09-18T01:08:57.622154+0000","last_clean":"2024-09-18T01:08:57.622154+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:08:57.622154+0000","last_undegraded":"2024-09-18T01:08:57.622154+0000","last_fullsized":"2024-09-18T01:08:57.622154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:19.924090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082562499999999997,"stat_sum":{"num_bytes":42944565,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":5585,"num_read_kb":45849,"num_write":263,"num_write_kb":21073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.14","version":"107'427","reported_seq":1669,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.589769+0000","last_change":"2024-09-18T01:08:51.568244+0000","last_active":"2024-09-18T01:08:55.589769+0000","last_peered":"2024-09-18T01:08:55.589769+0000","last_clean":"2024-09-18T01:08:55.589769+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:08:55.589769+0000","last_undegraded":"2024-09-18T01:08:55.589769+0000","last_fullsized":"2024-09-18T01:08:55.589769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:45.727728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00121653,"stat_sum":{"num_bytes":36584907,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2540,"num_read_kb":12438,"num_write":629,"num_write_kb":7473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.620804+0000","last_change":"2024-09-18T01:08:30.560455+0000","last_active":"2024-09-18T01:08:57.620804+0000","last_peered":"2024-09-18T01:08:57.620804+0000","last_clean":"2024-09-18T01:08:57.620804+0000","last_became_active":"2024-09-18T01:08:30.560107+0000","last_became_peered":"2024-09-18T01:08:30.560107+0000","last_unstale":"2024-09-18T01:08:57.620804+0000","last_undegraded":"2024-09-18T01:08:57.620804+0000","last_fullsized":"2024-09-18T01:08:57.620804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:34.598903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'228","reported_seq":1681,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.712096+0000","last_change":"2024-09-18T01:08:57.604461+0000","last_active":"2024-09-18T01:08:57.712096+0000","last_peered":"2024-09-18T01:08:57.712096+0000","last_clean":"2024-09-18T01:08:57.712096+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:08:57.712096+0000","last_undegraded":"2024-09-18T01:08:57.712096+0000","last_fullsized":"2024-09-18T01:08:57.712096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:36.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015081669999999999,"stat_sum":{"num_bytes":12941208,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2044,"num_read_kb":15099,"num_write":303,"num_write_kb":12035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599936+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:08:57.599936+0000","last_peered":"2024-09-18T01:08:57.599936+0000","last_clean":"2024-09-18T01:08:57.599936+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:08:57.599936+0000","last_undegraded":"2024-09-18T01:08:57.599936+0000","last_fullsized":"2024-09-18T01:08:57.599936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"107'230","reported_seq":2650,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.720153+0000","last_change":"2024-09-18T01:08:51.566010+0000","last_active":"2024-09-18T01:08:54.720153+0000","last_peered":"2024-09-18T01:08:54.720153+0000","last_clean":"2024-09-18T01:08:54.720153+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:08:54.720153+0000","last_undegraded":"2024-09-18T01:08:54.720153+0000","last_fullsized":"2024-09-18T01:08:54.720153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:40.836870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000465029,"stat_sum":{"num_bytes":25957017,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2672,"num_read_kb":20341,"num_write":378,"num_write_kb":14870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621345+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:08:57.621345+0000","last_peered":"2024-09-18T01:08:57.621345+0000","last_clean":"2024-09-18T01:08:57.621345+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:08:57.621345+0000","last_undegraded":"2024-09-18T01:08:57.621345+0000","last_fullsized":"2024-09-18T01:08:57.621345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"109'179","reported_seq":2043,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.600146+0000","last_change":"2024-09-18T01:08:57.600343+0000","last_active":"2024-09-18T01:08:57.600146+0000","last_peered":"2024-09-18T01:08:57.600146+0000","last_clean":"2024-09-18T01:08:57.600146+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:08:57.600146+0000","last_undegraded":"2024-09-18T01:08:57.600146+0000","last_fullsized":"2024-09-18T01:08:57.600146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:26.060712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072303899999999997,"stat_sum":{"num_bytes":38461258,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":1845,"num_read_kb":31820,"num_write":184,"num_write_kb":13415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599761+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:08:57.599761+0000","last_peered":"2024-09-18T01:08:57.599761+0000","last_clean":"2024-09-18T01:08:57.599761+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:08:57.599761+0000","last_undegraded":"2024-09-18T01:08:57.599761+0000","last_fullsized":"2024-09-18T01:08:57.599761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'327","reported_seq":8213,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608943+0000","last_change":"2024-09-18T01:08:57.609141+0000","last_active":"2024-09-18T01:08:57.608943+0000","last_peered":"2024-09-18T01:08:57.608943+0000","last_clean":"2024-09-18T01:08:57.608943+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:08:57.608943+0000","last_undegraded":"2024-09-18T01:08:57.608943+0000","last_fullsized":"2024-09-18T01:08:57.608943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:49.754200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000880012,"stat_sum":{"num_bytes":56937783,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":8277,"num_read_kb":50408,"num_write":557,"num_write_kb":27047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.3","version":"108'238","reported_seq":1488,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.610523+0000","last_change":"2024-09-18T01:08:51.568557+0000","last_active":"2024-09-18T01:08:55.610523+0000","last_peered":"2024-09-18T01:08:55.610523+0000","last_clean":"2024-09-18T01:08:55.610523+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:08:55.610523+0000","last_undegraded":"2024-09-18T01:08:55.610523+0000","last_fullsized":"2024-09-18T01:08:55.610523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:28.288256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444259,"stat_sum":{"num_bytes":15448757,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1685,"num_read_kb":16662,"num_write":315,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":198,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.607965+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:08:57.607965+0000","last_peered":"2024-09-18T01:08:57.607965+0000","last_clean":"2024-09-18T01:08:57.607965+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:08:57.607965+0000","last_undegraded":"2024-09-18T01:08:57.607965+0000","last_fullsized":"2024-09-18T01:08:57.607965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"109'174","reported_seq":7869,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599859+0000","last_change":"2024-09-18T01:08:57.600158+0000","last_active":"2024-09-18T01:08:57.599859+0000","last_peered":"2024-09-18T01:08:57.599859+0000","last_clean":"2024-09-18T01:08:57.599859+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:08:57.599859+0000","last_undegraded":"2024-09-18T01:08:57.599859+0000","last_fullsized":"2024-09-18T01:08:57.599859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:13.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000427556,"stat_sum":{"num_bytes":46635975,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":7595,"num_read_kb":47676,"num_write":329,"num_write_kb":20055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.598862+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:08:57.598862+0000","last_peered":"2024-09-18T01:08:57.598862+0000","last_clean":"2024-09-18T01:08:57.598862+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:08:57.598862+0000","last_undegraded":"2024-09-18T01:08:57.598862+0000","last_fullsized":"2024-09-18T01:08:57.598862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"109'290","reported_seq":7900,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.602844+0000","last_change":"2024-09-18T01:08:57.603077+0000","last_active":"2024-09-18T01:08:57.602844+0000","last_peered":"2024-09-18T01:08:57.602844+0000","last_clean":"2024-09-18T01:08:57.602844+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:08:57.602844+0000","last_undegraded":"2024-09-18T01:08:57.602844+0000","last_fullsized":"2024-09-18T01:08:57.602844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:39.086355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041808200000000001,"stat_sum":{"num_bytes":76481400,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":7781,"num_read_kb":24467,"num_write":314,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621071+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:08:57.621071+0000","last_peered":"2024-09-18T01:08:57.621071+0000","last_clean":"2024-09-18T01:08:57.621071+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:08:57.621071+0000","last_undegraded":"2024-09-18T01:08:57.621071+0000","last_fullsized":"2024-09-18T01:08:57.621071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"106'174","reported_seq":1455,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577402+0000","last_change":"2024-09-18T01:08:51.565690+0000","last_active":"2024-09-18T01:08:54.577402+0000","last_peered":"2024-09-18T01:08:54.577402+0000","last_clean":"2024-09-18T01:08:54.577402+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:08:54.577402+0000","last_undegraded":"2024-09-18T01:08:54.577402+0000","last_fullsized":"2024-09-18T01:08:54.577402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:34.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041185199999999999,"stat_sum":{"num_bytes":19872142,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1280,"num_read_kb":12789,"num_write":195,"num_write_kb":18372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.607905+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:08:57.607905+0000","last_peered":"2024-09-18T01:08:57.607905+0000","last_clean":"2024-09-18T01:08:57.607905+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:08:57.607905+0000","last_undegraded":"2024-09-18T01:08:57.607905+0000","last_fullsized":"2024-09-18T01:08:57.607905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":348,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.620970+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:08:57.620970+0000","last_peered":"2024-09-18T01:08:57.620970+0000","last_clean":"2024-09-18T01:08:57.620970+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:08:57.620970+0000","last_undegraded":"2024-09-18T01:08:57.620970+0000","last_fullsized":"2024-09-18T01:08:57.620970+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"107'526","reported_seq":1676,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577383+0000","last_change":"2024-09-18T01:08:51.565807+0000","last_active":"2024-09-18T01:08:54.577383+0000","last_peered":"2024-09-18T01:08:54.577383+0000","last_clean":"2024-09-18T01:08:54.577383+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:08:54.577383+0000","last_undegraded":"2024-09-18T01:08:54.577383+0000","last_fullsized":"2024-09-18T01:08:54.577383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:04.500752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074888400000000001,"stat_sum":{"num_bytes":18584324,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2813,"num_read_kb":12498,"num_write":962,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"108'124","reported_seq":5663,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.745502+0000","last_change":"2024-09-18T01:08:51.568356+0000","last_active":"2024-09-18T01:08:55.745502+0000","last_peered":"2024-09-18T01:08:55.745502+0000","last_clean":"2024-09-18T01:08:55.745502+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:08:55.745502+0000","last_undegraded":"2024-09-18T01:08:55.745502+0000","last_fullsized":"2024-09-18T01:08:55.745502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:00.912443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317508,"stat_sum":{"num_bytes":35571058,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5225,"num_read_kb":29395,"num_write":123,"num_write_kb":11230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.a","version":"110'235","reported_seq":7747,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.720159+0000","last_change":"2024-09-18T01:08:57.621913+0000","last_active":"2024-09-18T01:08:57.720159+0000","last_peered":"2024-09-18T01:08:57.720159+0000","last_clean":"2024-09-18T01:08:57.720159+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:08:57.720159+0000","last_undegraded":"2024-09-18T01:08:57.720159+0000","last_fullsized":"2024-09-18T01:08:57.720159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:49.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230339999999999,"stat_sum":{"num_bytes":58544900,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":7878,"num_read_kb":28740,"num_write":278,"num_write_kb":11553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.b","version":"106'255","reported_seq":1590,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577205+0000","last_change":"2024-09-18T01:08:51.565956+0000","last_active":"2024-09-18T01:08:54.577205+0000","last_peered":"2024-09-18T01:08:54.577205+0000","last_clean":"2024-09-18T01:08:54.577205+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:08:54.577205+0000","last_undegraded":"2024-09-18T01:08:54.577205+0000","last_fullsized":"2024-09-18T01:08:54.577205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:53.173150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042886299999999998,"stat_sum":{"num_bytes":17226346,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1867,"num_read_kb":20279,"num_write":353,"num_write_kb":9925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"109'559","reported_seq":2143,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608268+0000","last_change":"2024-09-18T01:08:57.608433+0000","last_active":"2024-09-18T01:08:57.608268+0000","last_peered":"2024-09-18T01:08:57.608268+0000","last_clean":"2024-09-18T01:08:57.608268+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:08:57.608268+0000","last_undegraded":"2024-09-18T01:08:57.608268+0000","last_fullsized":"2024-09-18T01:08:57.608268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:41.670958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033406899999999999,"stat_sum":{"num_bytes":18905820,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3261,"num_read_kb":18270,"num_write":1023,"num_write_kb":12462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.d","version":"109'167","reported_seq":11167,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621373+0000","last_change":"2024-09-18T01:08:57.621603+0000","last_active":"2024-09-18T01:08:57.621373+0000","last_peered":"2024-09-18T01:08:57.621373+0000","last_clean":"2024-09-18T01:08:57.621373+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:08:57.621373+0000","last_undegraded":"2024-09-18T01:08:57.621373+0000","last_fullsized":"2024-09-18T01:08:57.621373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:16.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040663199999999998,"stat_sum":{"num_bytes":58789609,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":10885,"num_read_kb":41703,"num_write":187,"num_write_kb":18335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"108'489","reported_seq":2651,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.707977+0000","last_change":"2024-09-18T01:08:57.622090+0000","last_active":"2024-09-18T01:08:57.707977+0000","last_peered":"2024-09-18T01:08:57.707977+0000","last_clean":"2024-09-18T01:08:57.707977+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:08:57.707977+0000","last_undegraded":"2024-09-18T01:08:57.707977+0000","last_fullsized":"2024-09-18T01:08:57.707977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:35.260395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000736847,"stat_sum":{"num_bytes":36395364,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":3337,"num_read_kb":42618,"num_write":762,"num_write_kb":29201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.f","version":"109'189","reported_seq":1458,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.622097+0000","last_change":"2024-09-18T01:08:57.622452+0000","last_active":"2024-09-18T01:08:57.622097+0000","last_peered":"2024-09-18T01:08:57.622097+0000","last_clean":"2024-09-18T01:08:57.622097+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:08:57.622097+0000","last_undegraded":"2024-09-18T01:08:57.622097+0000","last_fullsized":"2024-09-18T01:08:57.622097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:51.929039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000998229,"stat_sum":{"num_bytes":40333110,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":1230,"num_read_kb":14279,"num_write":270,"num_write_kb":10684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.10","version":"109'148","reported_seq":1033,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621066+0000","last_change":"2024-09-18T01:08:57.621285+0000","last_active":"2024-09-18T01:08:57.621066+0000","last_peered":"2024-09-18T01:08:57.621066+0000","last_clean":"2024-09-18T01:08:57.621066+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:08:57.621066+0000","last_undegraded":"2024-09-18T01:08:57.621066+0000","last_fullsized":"2024-09-18T01:08:57.621066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.651392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040560200000000002,"stat_sum":{"num_bytes":16737007,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":646,"num_read_kb":13998,"num_write":163,"num_write_kb":15008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.11","version":"109'193","reported_seq":1389,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621760+0000","last_change":"2024-09-18T01:08:57.622013+0000","last_active":"2024-09-18T01:08:57.621760+0000","last_peered":"2024-09-18T01:08:57.621760+0000","last_clean":"2024-09-18T01:08:57.621760+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:08:57.621760+0000","last_undegraded":"2024-09-18T01:08:57.621760+0000","last_fullsized":"2024-09-18T01:08:57.621760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:17:17.139815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075577499999999998,"stat_sum":{"num_bytes":57788483,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1027,"num_read_kb":10259,"num_write":302,"num_write_kb":11427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.12","version":"109'814","reported_seq":2542,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603855+0000","last_change":"2024-09-18T01:08:57.604063+0000","last_active":"2024-09-18T01:08:57.603855+0000","last_peered":"2024-09-18T01:08:57.603855+0000","last_clean":"2024-09-18T01:08:57.603855+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:08:57.603855+0000","last_undegraded":"2024-09-18T01:08:57.603855+0000","last_fullsized":"2024-09-18T01:08:57.603855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:30.935127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001176641,"stat_sum":{"num_bytes":18296628,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4855,"num_read_kb":19131,"num_write":1415,"num_write_kb":11941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.13","version":"109'500","reported_seq":1495,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621434+0000","last_change":"2024-09-18T01:08:57.621653+0000","last_active":"2024-09-18T01:08:57.621434+0000","last_peered":"2024-09-18T01:08:57.621434+0000","last_clean":"2024-09-18T01:08:57.621434+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:08:57.621434+0000","last_undegraded":"2024-09-18T01:08:57.621434+0000","last_fullsized":"2024-09-18T01:08:57.621434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:01.126490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045858200000000002,"stat_sum":{"num_bytes":27268260,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2645,"num_read_kb":6692,"num_write":775,"num_write_kb":7850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1074482534,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":108436,"num_read_kb":766718,"num_write":13915,"num_write_kb":505456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651272192,"data_stored":647715985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9461,"ondisk_log_size":9461,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161984,"kb_used_data":100268,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94209856,"statfs":{"total":96636764160,"available":96470892544,"internally_reserved":0,"allocated":102674432,"data_stored":101440860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62732,"kb_used_data":31544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309108,"statfs":{"total":96636764160,"available":96572526592,"internally_reserved":0,"allocated":32301056,"data_stored":31467818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147536,"kb_used_data":108024,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94224304,"statfs":{"total":96636764160,"available":96485687296,"internally_reserved":0,"allocated":110616576,"data_stored":109362505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69892,"kb_used_data":38704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301948,"statfs":{"total":96636764160,"available":96565194752,"internally_reserved":0,"allocated":39632896,"data_stored":38703465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97608,"kb_used_data":62260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274232,"statfs":{"total":96636764160,"available":96536813568,"internally_reserved":0,"allocated":63754240,"data_stored":62676657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":139824,"kb_used_data":94752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94232016,"statfs":{"total":96636764160,"available":96493584384,"internally_reserved":0,"allocated":97026048,"data_stored":95815208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166356,"kb_used_data":121268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94205484,"statfs":{"total":96636764160,"available":96466415616,"internally_reserved":0,"allocated":124178432,"data_stored":122880211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126264,"kb_used_data":86752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94245576,"statfs":{"total":96636764160,"available":96507469824,"internally_reserved":0,"allocated":88834048,"data_stored":87734034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,16,5,27,107,93],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":95141888,"data_stored":94610055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62459904,"data_stored":62061853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38330368,"data_stored":38082413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90624000,"data_stored":90163764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":123592704,"data_stored":122961459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":109322240,"data_stored":108747701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31010816,"data_stored":30853033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":100790272,"data_stored":100235707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:08:58.943 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:08:58.944 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:08:58.944 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:58.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:08:59.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:08:59.040 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:59.039+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:59.053 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:59.053 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:08:59.053 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:08:59.053 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:08:59.053 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:59.062 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:08:59.146 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:08:59.147 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:08:59.147 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:59.146+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:59.164 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:08:59.165 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:08:59.181 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:08:59.181 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:08:59.198 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:08:59.198 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:08:59.209 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:08:59.209 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:08:59.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:08:59.209 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:08:59.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.309784795, 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:08:59.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:08:59.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65805 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 124928~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5862879599999999, 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053947967999999999, 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65805 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621308+0000", 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0710000000000002e-06 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621315+0000", 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9840000000000004e-06 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621320+0000", 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675017+0000", 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053696902999999997 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675064+0000", 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7305999999999997e-05 2024-09-18T01:08:59.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675253+0000", 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018950400000000001 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.586267962, 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:08:59.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5862465510000003, 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:08:59.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:08:59.230 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5862236359999997, 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:08:59.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:08:59.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5862026189999998, 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:08:59.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5861714119999997, 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:08:59.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5861490839999997, 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.245 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:08:59.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5861270599999999, 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:08:59.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5860776330000004, 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:08:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:08:59.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5860465359999996, 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:08:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:08:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:08:59.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5860235410000003, 2024-09-18T01:08:59.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:08:59.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:08:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:08:59.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5860018230000001, 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:08:59.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5859804019999997, 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.273 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:08:59.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:08:59.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5859592449999997, 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:08:59.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5859342659999998, 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:08:59.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:08:59.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5859029429999998, 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:08:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5858800540000004, 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5858564250000002, 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:08:59.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:08:59.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5858342439999999, 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:08:59.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:08:59.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:08:59.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:08:59.305 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:08:59.308 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:08:59.309 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:08:59.309 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:08:59.310 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:08:59.310 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:59.306+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:59.310 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:08:59.311 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:08:59.311 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:08:59.311 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:08:59.312 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:08:59.313 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:08:59.313 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:08:59.315 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-18T01:08:58.739034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1075072921,"num_objects":1275,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1275,"num_whiteouts":0,"num_read":108553,"num_read_kb":766819,"num_write":14076,"num_write_kb":507874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9581,"ondisk_log_size":9581,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972196,"kb_used_data":643572,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754002524,"statfs":{"total":773094113280,"available":772098584576,"internally_reserved":0,"allocated":659017728,"data_stored":650080758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,16,5,27,107,93],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":128899442,"num_objects":470,"num_object_clones":0,"num_object_copies":940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":47938,"num_read_kb":86013,"num_write":1777,"num_write_kb":79950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000679"},"pg_stats":[{"pgid":"3.1f","version":"109'240","reported_seq":1887,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.602923+0000","last_change":"2024-09-18T01:08:57.603130+0000","last_active":"2024-09-18T01:08:57.602923+0000","last_peered":"2024-09-18T01:08:57.602923+0000","last_clean":"2024-09-18T01:08:57.602923+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:08:57.602923+0000","last_undegraded":"2024-09-18T01:08:57.602923+0000","last_fullsized":"2024-09-18T01:08:57.602923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:34.552259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042484100000000002,"stat_sum":{"num_bytes":29947159,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1896,"num_read_kb":20363,"num_write":264,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1e","version":"106'144","reported_seq":1000,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577315+0000","last_change":"2024-09-18T01:08:51.566309+0000","last_active":"2024-09-18T01:08:54.577315+0000","last_peered":"2024-09-18T01:08:54.577315+0000","last_clean":"2024-09-18T01:08:54.577315+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:08:54.577315+0000","last_undegraded":"2024-09-18T01:08:54.577315+0000","last_fullsized":"2024-09-18T01:08:54.577315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:46.316335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062472000000000005,"stat_sum":{"num_bytes":12934403,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":773,"num_read_kb":16830,"num_write":221,"num_write_kb":10618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"110'175","reported_seq":1537,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.609311+0000","last_change":"2024-09-18T01:08:57.609311+0000","last_active":"2024-09-18T01:08:57.609311+0000","last_peered":"2024-09-18T01:08:57.609311+0000","last_clean":"2024-09-18T01:08:57.609311+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:08:57.609311+0000","last_undegraded":"2024-09-18T01:08:57.609311+0000","last_fullsized":"2024-09-18T01:08:57.609311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:47.480704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000911122,"stat_sum":{"num_bytes":33777976,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1013,"num_read_kb":27852,"num_write":252,"num_write_kb":31132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1c","version":"109'311","reported_seq":1579,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608467+0000","last_change":"2024-09-18T01:08:57.608635+0000","last_active":"2024-09-18T01:08:57.608467+0000","last_peered":"2024-09-18T01:08:57.608467+0000","last_clean":"2024-09-18T01:08:57.608467+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:08:57.608467+0000","last_undegraded":"2024-09-18T01:08:57.608467+0000","last_fullsized":"2024-09-18T01:08:57.608467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:32.662596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061793899999999997,"stat_sum":{"num_bytes":30508630,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":1825,"num_read_kb":25229,"num_write":447,"num_write_kb":26223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1b","version":"109'901","reported_seq":3374,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603581+0000","last_change":"2024-09-18T01:08:57.603782+0000","last_active":"2024-09-18T01:08:57.603581+0000","last_peered":"2024-09-18T01:08:57.603581+0000","last_clean":"2024-09-18T01:08:57.603581+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:08:57.603581+0000","last_undegraded":"2024-09-18T01:08:57.603581+0000","last_fullsized":"2024-09-18T01:08:57.603581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:54.201618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007729,"stat_sum":{"num_bytes":28425303,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":6509,"num_read_kb":29048,"num_write":1351,"num_write_kb":20735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.1a","version":"107'162","reported_seq":1820,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.589912+0000","last_change":"2024-09-18T01:08:51.567312+0000","last_active":"2024-09-18T01:08:55.589912+0000","last_peered":"2024-09-18T01:08:55.589912+0000","last_clean":"2024-09-18T01:08:55.589912+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:08:55.589912+0000","last_undegraded":"2024-09-18T01:08:55.589912+0000","last_fullsized":"2024-09-18T01:08:55.589912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:19.119192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643299999999999,"stat_sum":{"num_bytes":18574012,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1618,"num_read_kb":21557,"num_write":218,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.19","version":"109'159","reported_seq":1599,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.715628+0000","last_change":"2024-09-18T01:08:57.600053+0000","last_active":"2024-09-18T01:08:57.715628+0000","last_peered":"2024-09-18T01:08:57.715628+0000","last_clean":"2024-09-18T01:08:57.715628+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:08:57.715628+0000","last_undegraded":"2024-09-18T01:08:57.715628+0000","last_fullsized":"2024-09-18T01:08:57.715628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:37.265835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043584400000000001,"stat_sum":{"num_bytes":35778665,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1149,"num_read_kb":19160,"num_write":189,"num_write_kb":14501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.18","version":"106'183","reported_seq":4721,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.750358+0000","last_change":"2024-09-18T01:08:51.566328+0000","last_active":"2024-09-18T01:08:54.750358+0000","last_peered":"2024-09-18T01:08:54.750358+0000","last_clean":"2024-09-18T01:08:54.750358+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:08:54.750358+0000","last_undegraded":"2024-09-18T01:08:54.750358+0000","last_fullsized":"2024-09-18T01:08:54.750358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:09.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012726899999999999,"stat_sum":{"num_bytes":42968641,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4268,"num_read_kb":35281,"num_write":220,"num_write_kb":16625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"109'315","reported_seq":1382,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603638+0000","last_change":"2024-09-18T01:08:57.603853+0000","last_active":"2024-09-18T01:08:57.603638+0000","last_peered":"2024-09-18T01:08:57.603638+0000","last_clean":"2024-09-18T01:08:57.603638+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:08:57.603638+0000","last_undegraded":"2024-09-18T01:08:57.603638+0000","last_fullsized":"2024-09-18T01:08:57.603638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:00:41.195936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098992299999999997,"stat_sum":{"num_bytes":34021606,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1551,"num_read_kb":21053,"num_write":482,"num_write_kb":15397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.16","version":"108'156","reported_seq":1256,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608210+0000","last_change":"2024-09-18T01:08:57.608418+0000","last_active":"2024-09-18T01:08:57.608210+0000","last_peered":"2024-09-18T01:08:57.608210+0000","last_clean":"2024-09-18T01:08:57.608210+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:08:57.608210+0000","last_undegraded":"2024-09-18T01:08:57.608210+0000","last_fullsized":"2024-09-18T01:08:57.608210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:57:19.715620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395764,"stat_sum":{"num_bytes":30840218,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":861,"num_read_kb":14934,"num_write":191,"num_write_kb":11808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.15","version":"109'249","reported_seq":5740,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.622154+0000","last_change":"2024-09-18T01:08:57.622360+0000","last_active":"2024-09-18T01:08:57.622154+0000","last_peered":"2024-09-18T01:08:57.622154+0000","last_clean":"2024-09-18T01:08:57.622154+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:08:57.622154+0000","last_undegraded":"2024-09-18T01:08:57.622154+0000","last_fullsized":"2024-09-18T01:08:57.622154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:19.924090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082562499999999997,"stat_sum":{"num_bytes":42944565,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":5585,"num_read_kb":45849,"num_write":263,"num_write_kb":21073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.14","version":"107'427","reported_seq":1669,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.589769+0000","last_change":"2024-09-18T01:08:51.568244+0000","last_active":"2024-09-18T01:08:55.589769+0000","last_peered":"2024-09-18T01:08:55.589769+0000","last_clean":"2024-09-18T01:08:55.589769+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:08:55.589769+0000","last_undegraded":"2024-09-18T01:08:55.589769+0000","last_fullsized":"2024-09-18T01:08:55.589769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:45.727728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00121653,"stat_sum":{"num_bytes":36584907,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2540,"num_read_kb":12438,"num_write":629,"num_write_kb":7473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.620804+0000","last_change":"2024-09-18T01:08:30.560455+0000","last_active":"2024-09-18T01:08:57.620804+0000","last_peered":"2024-09-18T01:08:57.620804+0000","last_clean":"2024-09-18T01:08:57.620804+0000","last_became_active":"2024-09-18T01:08:30.560107+0000","last_became_peered":"2024-09-18T01:08:30.560107+0000","last_unstale":"2024-09-18T01:08:57.620804+0000","last_undegraded":"2024-09-18T01:08:57.620804+0000","last_fullsized":"2024-09-18T01:08:57.620804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:34.598903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'228","reported_seq":1681,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.712096+0000","last_change":"2024-09-18T01:08:57.604461+0000","last_active":"2024-09-18T01:08:57.712096+0000","last_peered":"2024-09-18T01:08:57.712096+0000","last_clean":"2024-09-18T01:08:57.712096+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:08:57.712096+0000","last_undegraded":"2024-09-18T01:08:57.712096+0000","last_fullsized":"2024-09-18T01:08:57.712096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:36.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015081669999999999,"stat_sum":{"num_bytes":12941208,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2044,"num_read_kb":15099,"num_write":303,"num_write_kb":12035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599936+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:08:57.599936+0000","last_peered":"2024-09-18T01:08:57.599936+0000","last_clean":"2024-09-18T01:08:57.599936+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:08:57.599936+0000","last_undegraded":"2024-09-18T01:08:57.599936+0000","last_fullsized":"2024-09-18T01:08:57.599936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"107'230","reported_seq":2650,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.720153+0000","last_change":"2024-09-18T01:08:51.566010+0000","last_active":"2024-09-18T01:08:54.720153+0000","last_peered":"2024-09-18T01:08:54.720153+0000","last_clean":"2024-09-18T01:08:54.720153+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:08:54.720153+0000","last_undegraded":"2024-09-18T01:08:54.720153+0000","last_fullsized":"2024-09-18T01:08:54.720153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:40.836870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000465029,"stat_sum":{"num_bytes":25957017,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2672,"num_read_kb":20341,"num_write":378,"num_write_kb":14870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621345+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:08:57.621345+0000","last_peered":"2024-09-18T01:08:57.621345+0000","last_clean":"2024-09-18T01:08:57.621345+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:08:57.621345+0000","last_undegraded":"2024-09-18T01:08:57.621345+0000","last_fullsized":"2024-09-18T01:08:57.621345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"109'179","reported_seq":2043,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.600146+0000","last_change":"2024-09-18T01:08:57.600343+0000","last_active":"2024-09-18T01:08:57.600146+0000","last_peered":"2024-09-18T01:08:57.600146+0000","last_clean":"2024-09-18T01:08:57.600146+0000","last_became_active":"2024-09-18T01:07:28.968453+0000","last_became_peered":"2024-09-18T01:07:28.968453+0000","last_unstale":"2024-09-18T01:08:57.600146+0000","last_undegraded":"2024-09-18T01:08:57.600146+0000","last_fullsized":"2024-09-18T01:08:57.600146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:26.060712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072303899999999997,"stat_sum":{"num_bytes":38461258,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":1845,"num_read_kb":31820,"num_write":184,"num_write_kb":13415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599761+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:08:57.599761+0000","last_peered":"2024-09-18T01:08:57.599761+0000","last_clean":"2024-09-18T01:08:57.599761+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:08:57.599761+0000","last_undegraded":"2024-09-18T01:08:57.599761+0000","last_fullsized":"2024-09-18T01:08:57.599761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'327","reported_seq":8213,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608943+0000","last_change":"2024-09-18T01:08:57.609141+0000","last_active":"2024-09-18T01:08:57.608943+0000","last_peered":"2024-09-18T01:08:57.608943+0000","last_clean":"2024-09-18T01:08:57.608943+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:08:57.608943+0000","last_undegraded":"2024-09-18T01:08:57.608943+0000","last_fullsized":"2024-09-18T01:08:57.608943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:49.754200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000880012,"stat_sum":{"num_bytes":56937783,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":8277,"num_read_kb":50408,"num_write":557,"num_write_kb":27047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.3","version":"108'238","reported_seq":1488,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.610523+0000","last_change":"2024-09-18T01:08:51.568557+0000","last_active":"2024-09-18T01:08:55.610523+0000","last_peered":"2024-09-18T01:08:55.610523+0000","last_clean":"2024-09-18T01:08:55.610523+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:08:55.610523+0000","last_undegraded":"2024-09-18T01:08:55.610523+0000","last_fullsized":"2024-09-18T01:08:55.610523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:28.288256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444259,"stat_sum":{"num_bytes":15448757,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1685,"num_read_kb":16662,"num_write":315,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":198,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.607965+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:08:57.607965+0000","last_peered":"2024-09-18T01:08:57.607965+0000","last_clean":"2024-09-18T01:08:57.607965+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:08:57.607965+0000","last_undegraded":"2024-09-18T01:08:57.607965+0000","last_fullsized":"2024-09-18T01:08:57.607965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"109'174","reported_seq":7869,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.599859+0000","last_change":"2024-09-18T01:08:57.600158+0000","last_active":"2024-09-18T01:08:57.599859+0000","last_peered":"2024-09-18T01:08:57.599859+0000","last_clean":"2024-09-18T01:08:57.599859+0000","last_became_active":"2024-09-18T01:07:28.968437+0000","last_became_peered":"2024-09-18T01:07:28.968437+0000","last_unstale":"2024-09-18T01:08:57.599859+0000","last_undegraded":"2024-09-18T01:08:57.599859+0000","last_fullsized":"2024-09-18T01:08:57.599859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:13.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000427556,"stat_sum":{"num_bytes":46635975,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":7595,"num_read_kb":47676,"num_write":329,"num_write_kb":20055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.598862+0000","last_change":"2024-09-18T01:07:20.949080+0000","last_active":"2024-09-18T01:08:57.598862+0000","last_peered":"2024-09-18T01:08:57.598862+0000","last_clean":"2024-09-18T01:08:57.598862+0000","last_became_active":"2024-09-18T01:07:19.410230+0000","last_became_peered":"2024-09-18T01:07:19.410230+0000","last_unstale":"2024-09-18T01:08:57.598862+0000","last_undegraded":"2024-09-18T01:08:57.598862+0000","last_fullsized":"2024-09-18T01:08:57.598862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:29.435693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051079299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"109'290","reported_seq":7900,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.602844+0000","last_change":"2024-09-18T01:08:57.603077+0000","last_active":"2024-09-18T01:08:57.602844+0000","last_peered":"2024-09-18T01:08:57.602844+0000","last_clean":"2024-09-18T01:08:57.602844+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:08:57.602844+0000","last_undegraded":"2024-09-18T01:08:57.602844+0000","last_fullsized":"2024-09-18T01:08:57.602844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:39.086355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041808200000000001,"stat_sum":{"num_bytes":76481400,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":7781,"num_read_kb":24467,"num_write":314,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621071+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:08:57.621071+0000","last_peered":"2024-09-18T01:08:57.621071+0000","last_clean":"2024-09-18T01:08:57.621071+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:08:57.621071+0000","last_undegraded":"2024-09-18T01:08:57.621071+0000","last_fullsized":"2024-09-18T01:08:57.621071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"106'174","reported_seq":1455,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577402+0000","last_change":"2024-09-18T01:08:51.565690+0000","last_active":"2024-09-18T01:08:54.577402+0000","last_peered":"2024-09-18T01:08:54.577402+0000","last_clean":"2024-09-18T01:08:54.577402+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:08:54.577402+0000","last_undegraded":"2024-09-18T01:08:54.577402+0000","last_fullsized":"2024-09-18T01:08:54.577402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:34.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041185199999999999,"stat_sum":{"num_bytes":19872142,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1280,"num_read_kb":12789,"num_write":195,"num_write_kb":18372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.607905+0000","last_change":"2024-09-18T01:07:20.952512+0000","last_active":"2024-09-18T01:08:57.607905+0000","last_peered":"2024-09-18T01:08:57.607905+0000","last_clean":"2024-09-18T01:08:57.607905+0000","last_became_active":"2024-09-18T01:07:18.938919+0000","last_became_peered":"2024-09-18T01:07:18.938919+0000","last_unstale":"2024-09-18T01:08:57.607905+0000","last_undegraded":"2024-09-18T01:08:57.607905+0000","last_fullsized":"2024-09-18T01:08:57.607905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:56.444894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":348,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.620970+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:08:57.620970+0000","last_peered":"2024-09-18T01:08:57.620970+0000","last_clean":"2024-09-18T01:08:57.620970+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:08:57.620970+0000","last_undegraded":"2024-09-18T01:08:57.620970+0000","last_fullsized":"2024-09-18T01:08:57.620970+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"107'526","reported_seq":1676,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577383+0000","last_change":"2024-09-18T01:08:51.565807+0000","last_active":"2024-09-18T01:08:54.577383+0000","last_peered":"2024-09-18T01:08:54.577383+0000","last_clean":"2024-09-18T01:08:54.577383+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:08:54.577383+0000","last_undegraded":"2024-09-18T01:08:54.577383+0000","last_fullsized":"2024-09-18T01:08:54.577383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:04.500752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074888400000000001,"stat_sum":{"num_bytes":18584324,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2813,"num_read_kb":12498,"num_write":962,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"108'124","reported_seq":5663,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T01:08:55.745502+0000","last_change":"2024-09-18T01:08:51.568356+0000","last_active":"2024-09-18T01:08:55.745502+0000","last_peered":"2024-09-18T01:08:55.745502+0000","last_clean":"2024-09-18T01:08:55.745502+0000","last_became_active":"2024-09-18T01:07:28.968577+0000","last_became_peered":"2024-09-18T01:07:28.968577+0000","last_unstale":"2024-09-18T01:08:55.745502+0000","last_undegraded":"2024-09-18T01:08:55.745502+0000","last_fullsized":"2024-09-18T01:08:55.745502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:00.912443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317508,"stat_sum":{"num_bytes":35571058,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5225,"num_read_kb":29395,"num_write":123,"num_write_kb":11230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.a","version":"110'235","reported_seq":7747,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.720159+0000","last_change":"2024-09-18T01:08:57.621913+0000","last_active":"2024-09-18T01:08:57.720159+0000","last_peered":"2024-09-18T01:08:57.720159+0000","last_clean":"2024-09-18T01:08:57.720159+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:08:57.720159+0000","last_undegraded":"2024-09-18T01:08:57.720159+0000","last_fullsized":"2024-09-18T01:08:57.720159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:49.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230339999999999,"stat_sum":{"num_bytes":58544900,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":7878,"num_read_kb":28740,"num_write":278,"num_write_kb":11553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.b","version":"106'255","reported_seq":1590,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-18T01:08:54.577205+0000","last_change":"2024-09-18T01:08:51.565956+0000","last_active":"2024-09-18T01:08:54.577205+0000","last_peered":"2024-09-18T01:08:54.577205+0000","last_clean":"2024-09-18T01:08:54.577205+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:08:54.577205+0000","last_undegraded":"2024-09-18T01:08:54.577205+0000","last_fullsized":"2024-09-18T01:08:54.577205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:53.173150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042886299999999998,"stat_sum":{"num_bytes":17226346,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1867,"num_read_kb":20279,"num_write":353,"num_write_kb":9925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"109'559","reported_seq":2143,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.608268+0000","last_change":"2024-09-18T01:08:57.608433+0000","last_active":"2024-09-18T01:08:57.608268+0000","last_peered":"2024-09-18T01:08:57.608268+0000","last_clean":"2024-09-18T01:08:57.608268+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:08:57.608268+0000","last_undegraded":"2024-09-18T01:08:57.608268+0000","last_fullsized":"2024-09-18T01:08:57.608268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:41.670958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033406899999999999,"stat_sum":{"num_bytes":18905820,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3261,"num_read_kb":18270,"num_write":1023,"num_write_kb":12462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.d","version":"109'167","reported_seq":11167,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621373+0000","last_change":"2024-09-18T01:08:57.621603+0000","last_active":"2024-09-18T01:08:57.621373+0000","last_peered":"2024-09-18T01:08:57.621373+0000","last_clean":"2024-09-18T01:08:57.621373+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:08:57.621373+0000","last_undegraded":"2024-09-18T01:08:57.621373+0000","last_fullsized":"2024-09-18T01:08:57.621373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:16.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040663199999999998,"stat_sum":{"num_bytes":58789609,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":10885,"num_read_kb":41703,"num_write":187,"num_write_kb":18335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"108'489","reported_seq":2651,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.707977+0000","last_change":"2024-09-18T01:08:57.622090+0000","last_active":"2024-09-18T01:08:57.707977+0000","last_peered":"2024-09-18T01:08:57.707977+0000","last_clean":"2024-09-18T01:08:57.707977+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:08:57.707977+0000","last_undegraded":"2024-09-18T01:08:57.707977+0000","last_fullsized":"2024-09-18T01:08:57.707977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:35.260395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000736847,"stat_sum":{"num_bytes":36395364,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":3337,"num_read_kb":42618,"num_write":762,"num_write_kb":29201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.f","version":"109'189","reported_seq":1458,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.622097+0000","last_change":"2024-09-18T01:08:57.622452+0000","last_active":"2024-09-18T01:08:57.622097+0000","last_peered":"2024-09-18T01:08:57.622097+0000","last_clean":"2024-09-18T01:08:57.622097+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:08:57.622097+0000","last_undegraded":"2024-09-18T01:08:57.622097+0000","last_fullsized":"2024-09-18T01:08:57.622097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:51.929039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000998229,"stat_sum":{"num_bytes":40333110,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":1230,"num_read_kb":14279,"num_write":270,"num_write_kb":10684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.10","version":"109'148","reported_seq":1033,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621066+0000","last_change":"2024-09-18T01:08:57.621285+0000","last_active":"2024-09-18T01:08:57.621066+0000","last_peered":"2024-09-18T01:08:57.621066+0000","last_clean":"2024-09-18T01:08:57.621066+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:08:57.621066+0000","last_undegraded":"2024-09-18T01:08:57.621066+0000","last_fullsized":"2024-09-18T01:08:57.621066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.651392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040560200000000002,"stat_sum":{"num_bytes":16737007,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":646,"num_read_kb":13998,"num_write":163,"num_write_kb":15008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.11","version":"109'193","reported_seq":1389,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621760+0000","last_change":"2024-09-18T01:08:57.622013+0000","last_active":"2024-09-18T01:08:57.621760+0000","last_peered":"2024-09-18T01:08:57.621760+0000","last_clean":"2024-09-18T01:08:57.621760+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:08:57.621760+0000","last_undegraded":"2024-09-18T01:08:57.621760+0000","last_fullsized":"2024-09-18T01:08:57.621760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:17:17.139815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075577499999999998,"stat_sum":{"num_bytes":57788483,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1027,"num_read_kb":10259,"num_write":302,"num_write_kb":11427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.12","version":"109'814","reported_seq":2542,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.603855+0000","last_change":"2024-09-18T01:08:57.604063+0000","last_active":"2024-09-18T01:08:57.603855+0000","last_peered":"2024-09-18T01:08:57.603855+0000","last_clean":"2024-09-18T01:08:57.603855+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:08:57.603855+0000","last_undegraded":"2024-09-18T01:08:57.603855+0000","last_fullsized":"2024-09-18T01:08:57.603855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:30.935127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001176641,"stat_sum":{"num_bytes":18296628,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4855,"num_read_kb":19131,"num_write":1415,"num_write_kb":11941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.13","version":"109'500","reported_seq":1495,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-18T01:08:57.621434+0000","last_change":"2024-09-18T01:08:57.621653+0000","last_active":"2024-09-18T01:08:57.621434+0000","last_peered":"2024-09-18T01:08:57.621434+0000","last_clean":"2024-09-18T01:08:57.621434+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:08:57.621434+0000","last_undegraded":"2024-09-18T01:08:57.621434+0000","last_fullsized":"2024-09-18T01:08:57.621434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:01.126490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045858200000000002,"stat_sum":{"num_bytes":27268260,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2645,"num_read_kb":6692,"num_write":775,"num_write_kb":7850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1074482534,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":108436,"num_read_kb":766718,"num_write":13915,"num_write_kb":505456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651272192,"data_stored":647715985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9461,"ondisk_log_size":9461,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161984,"kb_used_data":100268,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94209856,"statfs":{"total":96636764160,"available":96470892544,"internally_reserved":0,"allocated":102674432,"data_stored":101440860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62732,"kb_used_data":31544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309108,"statfs":{"total":96636764160,"available":96572526592,"internally_reserved":0,"allocated":32301056,"data_stored":31467818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147536,"kb_used_data":108024,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94224304,"statfs":{"total":96636764160,"available":96485687296,"internally_reserved":0,"allocated":110616576,"data_stored":109362505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69892,"kb_used_data":38704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301948,"statfs":{"total":96636764160,"available":96565194752,"internally_reserved":0,"allocated":39632896,"data_stored":38703465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97608,"kb_used_data":62260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274232,"statfs":{"total":96636764160,"available":96536813568,"internally_reserved":0,"allocated":63754240,"data_stored":62676657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":139824,"kb_used_data":94752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94232016,"statfs":{"total":96636764160,"available":96493584384,"internally_reserved":0,"allocated":97026048,"data_stored":95815208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166356,"kb_used_data":121268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94205484,"statfs":{"total":96636764160,"available":96466415616,"internally_reserved":0,"allocated":124178432,"data_stored":122880211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126264,"kb_used_data":86752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94245576,"statfs":{"total":96636764160,"available":96507469824,"internally_reserved":0,"allocated":88834048,"data_stored":87734034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,16,5,27,107,93],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":95141888,"data_stored":94610055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62459904,"data_stored":62061853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38330368,"data_stored":38082413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90624000,"data_stored":90163764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":123592704,"data_stored":122961459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":109322240,"data_stored":108747701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31010816,"data_stored":30853033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":100790272,"data_stored":100235707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:08:59.317 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:08:59.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:08:59.317 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:08:59.318 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-18T01:08:59.408 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:59.407+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:59.460 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:59.461 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:59.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:08:59.511 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:08:59.510+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:08:59.610 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:08:59.608+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:08:59.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:08:59.626 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:59.626 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:59.636 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:08:59.663 INFO:tasks.ceph.mon.a.smithi120.stderr:2024-09-18T01:08:59.662+0000 7f77b87d3640 -1 mon.a@0(leader).osd e112 definitely_dead 0 2024-09-18T01:08:59.710 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:08:59.709+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:08:59.770 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:59.770 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:59.779 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:08:59.820 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:08:59.819+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:08:59.913 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:08:59.912+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:08:59.925 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:08:59.925 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:08:59.933 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:00.014 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:00.013+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:00.036 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.036 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.044 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:00.129 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:00.128+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:00.187 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.187 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:00.215 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:00.213+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:00.316 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:00.315+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:00.338 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.338 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.347 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:00.433 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:00.432+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:00.490 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.491 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.499 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:00.517 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:00.516+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:00.606 INFO:teuthology.orchestra.run.smithi120.stderr:marked out osd.2. 2024-09-18T01:09:00.612 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:09:00.618 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:00.617+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:00.647 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-18T01:09:00.652 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.652 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:00.719 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:00.717+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:00.808 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.808 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:00.820 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:00.819+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:00.919 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:00.919 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:00.920 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:00.919+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:00.927 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:01.021 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:01.019+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:01.071 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:01.071 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:01.079 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:01.121 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:01.121+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:01.196 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:01.196 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:01.206 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:01.250 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:01.248+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:01.308 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:01.308 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:01.316 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:01.317 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:01.360 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:01.359+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:01.423 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:01.421+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:01.435 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:01.436 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:01.453 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:01.461 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:01.461 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:01.468 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:01.470 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:01.483 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:01.483 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:01.500 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:01.500 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:01.517 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:01.517 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:01.534 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:01.534 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:01.550 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:01.550 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:01.554 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:01.552+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:01.619 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:01.619 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:01.619 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:01.619 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:01.625 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:01.623+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:01.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:01.726 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:01.725+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:01.772 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:01.772 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:01.772 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:01.772 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:01.772 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:01.781 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:01.826 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:01.824+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:01.885 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860562005999995, 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.886 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:01.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860519855000007, 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:01.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:01.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:01.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860486426999998, 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.896 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:01.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:01.901 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860448926999993, 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:01.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860427349999995, 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:01.906 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:01.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860408210000003, 2024-09-18T01:09:01.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:01.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:01.916 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860387818999996, 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:01.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:01.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860358418999994, 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:01.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:01.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860329629000006, 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:01.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:01.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860308934000003, 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:01.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:01.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860289449999996, 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:01.940 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.941 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:01.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860255538999994, 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:01.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.860219540000003, 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:01.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.775771567999996, 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:01.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:01.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.775761359000001, 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:01.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:01.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:01.965 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.275360133000007, 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:01.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:01.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.259873963999993, 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:01.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:01.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:01.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:01.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:01.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:01.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:09:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.251983244000002, 2024-09-18T01:09:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:09:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:09:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:09:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:09:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:09:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:09:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:09:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:09:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:09:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:09:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:09:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.225914240999998, 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.220745449000006, 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.005 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:02.008 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:02.006+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:02.108 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:02.107+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:02.151 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:02.151 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.160 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:02.209 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:02.207+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:02.263 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.263 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:02.263 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:02.264 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:02.264 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.273 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:02.310 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:02.309+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:02.378 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.378 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:02.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:02.378 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:02.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.805286930999998, 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:02.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:02.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:02.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.753295788000003, 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:02.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:02.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.753218992000001, 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:02.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:02.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.748713838999997, 2024-09-18T01:09:02.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:02.397 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.398 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.398 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:02.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:02.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:02.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:02.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:02.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:02.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:02.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.408 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.748458042999999, 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.409 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.410 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:02.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:02.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:02.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:02.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:02.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:02.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:02.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.744432715000002, 2024-09-18T01:09:02.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.744291564999997, 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:02.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:02.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.744058660999997, 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.431 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:02.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.741439599000003, 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.436 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:02.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:02.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.719100564999998, 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:02.441 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:02.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7685852910000008, 2024-09-18T01:09:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7685635499999997, 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:02.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7685431420000004, 2024-09-18T01:09:02.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:02.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7685234829999992, 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7684901479999997, 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7684556160000007, 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7684107260000008, 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7683843509999999, 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7683651999999999, 2024-09-18T01:09:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7683466320000001, 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.509 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:02.511 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:02.509+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:02.611 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:02.610+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:02.617 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.617 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:02.618 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:02.618 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.627 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:02.633 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:09:02.711 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:02.709+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:02.742 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:02.743 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:02.743 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:02.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:02.839 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:02.837+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:02.903 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:02.904 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:02.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:02.904 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:02.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 111.003218191, 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:02.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:02.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:02.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:02.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65805 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 124928~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2797213559999996, 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053947967999999999, 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65805 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:02.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621308+0000", 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0710000000000002e-06 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621315+0000", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9840000000000004e-06 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621320+0000", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675017+0000", 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053696902999999997 2024-09-18T01:09:02.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675064+0000", 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7305999999999997e-05 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675253+0000", 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018950400000000001 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2797013580000005, 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:02.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.279679947, 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:02.923 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.927 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2796570319999994, 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:02.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2796360149999995, 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:02.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:02.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2796048080000002, 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:02.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2795824800000002, 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:02.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:02.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2795604560000005, 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:02.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:02.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:02.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.279511029, 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:02.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2794799319999992, 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:02.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:02.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2794569370000008, 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:02.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2794352189999998, 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2794137980000002, 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.973 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:02.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:02.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2793926409999994, 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:02.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:02.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2793676620000003, 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2793363390000003, 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:02.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:02.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2793134500000001, 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:02.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:02.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:02.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2792898210000008, 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:02.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:03.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:03.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2792676400000005, 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:03.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:03.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:03.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:03.007 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:03.009 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:03.007+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:03.009 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:03.109 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:03.107+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:03.129 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:03.129 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:03.129 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:03.130 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:03.139 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:03.171 INFO:teuthology.orchestra.run.smithi120.stdout:2557 read 0x1a5e thru 0x742a (0x59cd bytes) 2024-09-18T01:09:03.172 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:03.172 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:03.182 INFO:teuthology.orchestra.run.smithi120.stdout:2561 trunc from 0x7735 to 0x887f8f0 2024-09-18T01:09:03.194 INFO:teuthology.orchestra.run.smithi120.stdout:2564 trunc from 0x887f8f0 to 0xa04d35e 2024-09-18T01:09:03.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:03.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:03.210 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:03.208+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:03.248 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:03.248 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:03.248 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:03.248 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:03.248 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:03.257 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:03.341 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:03.340+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.803686026999998, 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.742083405000002, 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 65.677234017000004, 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4357593790000003, 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4357352139999993, 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4357141060000007, 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4356932330000003, 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4356750070000004, 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4355828370000001, 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4355544259999995, 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4355324350000007, 2024-09-18T01:09:03.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:03.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4355124539999995, 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.463 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4354915970000004, 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4307159570000003, 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:03.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:03.472 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4306521409999995, 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:03.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4306268969999998, 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4305881500000002, 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:03.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4305658349999995, 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:03.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:03.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:03.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4305433250000004, 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:03.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:03.495 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.430507016, 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:03.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:03.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:03.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:03.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:03.500 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:03.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:03.502 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:03.500+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:03.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:03.622 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:09:03.622 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:03.621+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:03.653 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:03.653 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:03.653 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:03.653 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:03.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:03.704 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:03.703+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:03.709 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:03.710 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:03.728 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:03.728 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:03.744 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:03.744 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:03.760 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:03.761 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:03.777 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:03.777 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:03.794 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:03.795 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:03.803 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:03.802+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:03.808 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:03.808 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:03.810 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:03.811 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:03.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:03.828 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:03.828 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:03.904 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:03.903+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:03.930 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:03.930 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:03.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:04.005 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:04.004+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:04.043 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.043 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:04.136 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:04.135+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:04.195 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.195 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:04.245 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:04.244+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:04.308 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:04.307+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:04.347 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.347 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:04.409 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:04.408+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:04.459 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.459 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:04.510 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:04.508+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:04.579 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.579 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.588 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:04.632 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:04.631+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:04.691 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.691 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.700 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:04.741 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:04.740+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:04.813 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:04.812+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:04.818 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.819 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.828 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:04.914 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:04.912+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:04.932 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:04.932 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:04.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:05.015 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:05.015+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:05.084 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.084 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.092 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:05.116 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:05.116+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:05.196 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.196 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.204 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:05.248 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:05.246+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:05.307 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.307 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:05.317 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:05.317+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:05.419 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:05.418+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:05.459 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.459 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.468 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:05.520 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:05.519+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:05.571 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.572 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:05.621 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:05.620+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:05.628 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:05.629 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:05.629 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-18T01:09:05.629 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-18T01:09:05.636 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:09:05.684 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.684 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:05.722 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:05.721+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:05.776 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:05.777 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:05.777 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-18T01:09:05.798 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.798 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.807 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:05.840 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:05.852 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:05.851+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:05.910 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:05.910 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:05.918 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:05.923 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:05.923+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:05.952 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:05.953 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:05.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:05.955 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:05.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-18T01:09:05.984 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:06.001 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:06.017 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:06.018 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:06.025 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:06.024+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:06.033 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:06.034 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:06.050 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:06.050 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:06.063 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:06.063 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:06.067 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:06.067 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:06.072 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:06.084 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:06.084 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:06.101 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:06.101 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:06.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:06.120 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:06.120 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-18T01:09:06.161 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:06.160+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:06.177 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 114.27783177400001, 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65805 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 124928~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554334939, 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053947967999999999, 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65805 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621308+0000", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0710000000000002e-06 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621315+0000", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9840000000000004e-06 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621320+0000", 2024-09-18T01:09:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675017+0000", 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053696902999999997 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675064+0000", 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7305999999999997e-05 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675253+0000", 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018950400000000001 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554314940999999, 2024-09-18T01:09:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554293530000001, 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554270615, 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554249598, 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554218390999999, 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554196062999999, 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:06.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554174038999999, 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554124612000001, 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554093515, 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:06.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.55407052, 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:06.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:06.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554048802000001, 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:06.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554027381000001, 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.554006224, 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:06.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:06.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.553981244999999, 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.553949921999999, 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.553927033000001, 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.553903404, 2024-09-18T01:09:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.553881222999999, 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:06.293 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:06.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:06.296 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:06.297 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:06.297 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-18T01:09:06.297 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:06.293+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:06.395 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:06.394+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:06.440 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:06.440 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:06.440 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:06.441 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:06.451 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:06.473 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:06.474 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:06.474 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:06.476 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:06.476 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:06.476 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:06.476 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:06.477 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:06.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-18T01:09:06.516 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:06.515+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:06.556 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:06.557 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:06.557 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:06.557 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:06.557 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:06.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:06.597 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:06.596+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:06.656 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:06.657 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:06.657 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-18T01:09:06.698 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:06.697+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.117682861000006, 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:06.716 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:06.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:06.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:06.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.056080238999996, 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:06.723 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:06.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:06.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:06.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.991230850999997, 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:06.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:06.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749756213, 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:06.736 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:06.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749732048, 2024-09-18T01:09:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:06.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:06.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.74971094, 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:06.748 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:06.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749690067, 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:06.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:06.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749671841, 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:06.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749579670999999, 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.74955126, 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.768 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749529269, 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:06.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749509288, 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:06.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.749488431, 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:06.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744712791, 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:06.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:06.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744648975, 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:06.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744623731000001, 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:06.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:06.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744584983999999, 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.802 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:06.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:06.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744562669, 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:06.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:06.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744540159, 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.812 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.744503849999999, 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:06.816 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:06.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:06.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:06.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.821 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:06.821 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:06.821 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:06.822 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:06.823 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:06.821+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:06.828 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:06.828 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:06.828 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:06.828 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:06.828 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:06.829 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:06.831 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:06.832 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:06.923 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:06.922+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:06.931 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:06.931 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:06.940 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:07.024 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:07.023+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:07.125 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:07.124+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:07.196 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.015911943999996, 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:07.204 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.007129995, 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:07.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:07.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:07.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:07.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.211 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417605065, 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.212 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:07.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:07.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417582763, 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:07.217 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:07.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417561809, 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:07.222 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:07.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:07.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417540966000001, 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:07.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417519563999999, 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:07.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417500384, 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.239 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:07.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417481028999999, 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:07.244 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.245 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.245 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.245 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:07.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417442425000001, 2024-09-18T01:09:07.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:07.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:07.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417412285999999, 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:07.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417388833, 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.262 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:07.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417366817, 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:07.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.272 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417345849, 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:07.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417324990999999, 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.278 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:07.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:07.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:07.283 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417296542000001, 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417274401, 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:07.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:07.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:07.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417251046000001, 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.297 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:07.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.417228112, 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.414189113999999, 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:07.308 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.313 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:07.315 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:07.313+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:07.416 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:07.415+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:07.419 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.419 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:07.419 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:07.419 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.428 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:07.517 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:07.516+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:07.531 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:07.532 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:07.625 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:07.624+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:07.686 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.686 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:07.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:07.686 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 108.744853347, 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:07.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:07.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:09:07.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:09:07.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.476658030999999, 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:09:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:09:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.475062283, 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:09:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:09:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:09:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:09:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:09:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.474251663999993, 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:09:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.509517709999997, 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:09:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.506861006999998, 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.924213464000005, 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:09:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.509900005000006, 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:09:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.008200983000002, 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:09:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.006624096000003, 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:09:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 56.976876505999996, 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:09:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 56.821497020000002, 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:09:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 56.817843590000003, 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.102176915000001, 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:09:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 866 entries 100'101 (0'0) error 3:9153dd03:::rbd_data.117649d3c4d0.0000000000000093:head by client.4470.0:61161 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'99)", 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.255953741999999, 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048029769999999999, 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428633+0000", 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7910000000000005e-06 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428653+0000", 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109999999999999e-05 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428667+0000", 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4187e-05 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428764+0000", 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.433427+0000", 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046639020000000002 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 869 entries 100'103 (0'0) error 3:94e57bb7:::rbd_data.117649d3c4d0.00000000000000b2:head by client.4470.0:61192 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.242666964000001, 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047167169999999996, 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441912+0000", 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441920+0000", 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2260000000000005e-06 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441942+0000", 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1804e-05 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441957+0000", 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4599e-05 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.442024+0000", 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7860000000000004e-05 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.446628+0000", 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046035479999999998 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 871 entries 100'104 (0'0) error 3:90b555ad:::rbd_data.117649d3c4d0.00000000000000ba:head by client.4470.0:61200 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.240140045, 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049806420000000004, 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444439+0000", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444445+0000", 2024-09-18T01:09:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444459+0000", 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4241e-05 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444468+0000", 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5399999999999996e-06 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444541+0000", 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.449419+0000", 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048779060000000004 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116214 3.4 e110/23 3:2420fb28:::rbd_data.11766701b08b.0000000000000277:head v 110'180, mlcod=108'178)", 2024-09-18T01:09:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.6294906919999992, 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045364760000000002, 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999997e-07 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055097+0000", 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.208e-06 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055111+0000", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545999999999999e-05 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055118+0000", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.934e-06 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055175+0000", 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7240000000000001e-05 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055201+0000", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5978000000000001e-05 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059539+0000", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043376889999999996 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059624+0000", 2024-09-18T01:09:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5118999999999996e-05 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116558 3.9 e110/23 3:9119117f:::rbd_data.11766701b08b.0000000000000127:head v 110'143, mlcod=110'141)", 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4766727159999995, 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005246901, 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207914+0000", 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9929999999999995e-06 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207925+0000", 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0828e-05 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207934+0000", 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0860000000000008e-06 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207976+0000", 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2274000000000002e-05 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207991+0000", 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4296e-05 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213113+0000", 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051225649999999999 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213152+0000", 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9116999999999998e-05 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116699 3.4 e110/23 3:27f7a2f6:::rbd_data.11766701b08b.00000000000001bf:head v 110'196, mlcod=110'193)", 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.4180503210000008, 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042393159999999999, 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266529+0000", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266534+0000", 2024-09-18T01:09:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5500000000000002e-06 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266545+0000", 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0926e-05 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266551+0000", 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3049999999999998e-06 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266596+0000", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4864999999999998e-05 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266617+0000", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0682e-05 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270721+0000", 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041039140000000002 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270767+0000", 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.634e-05 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:07.779 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.781 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:07.781 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:07.779+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:07.882 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:07.881+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:07.924 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:07.924 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:07.933 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:08.017 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:08.016+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:08.075 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:08.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:08.076 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:08.076 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:08.076 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:08.083 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:08.082+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:08.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:08.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:08.184 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:08.183+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:08.230 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600878765999999, 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.231 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:08.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600853781, 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:08.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:08.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:08.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:08.240 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600834001999999, 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.6008076, 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:08.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600786104000001, 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:08.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.255 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600766807999999, 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:08.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.260 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600748974, 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:08.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600722911, 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:08.265 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:08.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600703555999999, 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:08.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:08.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:08.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600673312, 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:08.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600654716999999, 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:08.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600636158, 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:08.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:08.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600618104, 2024-09-18T01:09:08.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:08.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:08.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600600345, 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600574076000001, 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.299 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600555456, 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.303 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:08.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.600537206, 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.312 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.599985831, 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:08.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.321 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.321 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:08.321 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.598386845, 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:08.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.594620858000001, 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:08.326 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:08.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:08.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:08.330 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:08.332 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:08.333 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:08.333 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:08.334 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:08.334 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:08.335 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:08.335 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:08.336 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:08.331+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:08.344 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:08.361 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:08.432 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:08.431+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:08.452 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:08.453 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:08.462 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:08.546 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:08.545+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:08.606 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:08.606 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:08.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:08.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:09:08.656 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:08.655+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:08.736 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:08.734+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:08.760 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:08.760 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:08.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:08.853 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:08.852+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:08.912 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:08.913 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:08.921 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:08.963 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:08.961+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:09.037 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:09.036+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:09.065 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.065 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:09.138 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:09.137+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:09.177 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.177 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.185 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:09.239 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:09.238+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:09.329 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.329 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.338 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:09.339 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:09.338+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:09.441 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:09.440+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:09.441 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.441 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.450 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:09.545 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:09.544+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:09.594 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.594 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.603 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:09.641 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:09:09.646 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:09.644+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:09.746 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.747 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.747 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:09.745+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:09.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:09.847 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:09.846+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:09.858 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.858 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.867 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:09.950 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:09.949+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:09.970 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:09.970 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:09.979 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:10.052 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:10.050+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:10.086 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:10.086 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:10.094 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:10.153 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:10.152+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:10.237 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:10.237 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:10.246 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:10.290 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:10.288+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:10.349 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:10.349 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:10.358 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:10.358 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:10.356+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:10.373 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:10.458 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:10.456+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:10.505 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:10.506 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:10.515 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:10.521 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:10.521 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:10.538 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:10.538 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:10.555 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:10.556 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:10.561 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:10.560+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:10.572 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:10.572 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:10.588 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:10.589 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:10.606 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:10.606 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:10.623 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:10.623 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:10.626 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:10.626 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:10.635 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:10.639 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:10.639 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:10.663 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:10.661+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.563835796999996, 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.555053848, 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965528918, 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965506615999999, 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965485661999999, 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965464819000001, 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965443416999999, 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965424237000001, 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965404882000001, 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965366278000001, 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965336139000001, 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965312686000001, 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965290670000002, 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965269702000001, 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965248844000001, 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965220394999999, 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965198254000001, 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965174899000001, 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.965151965, 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.962112966999999, 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:10.842 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:10.844 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:10.844 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:10.842+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:10.945 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:10.944+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:10.987 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:10.987 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:10.995 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:11.046 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:11.044+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:11.098 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:11.098 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:11.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:11.147 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:11.146+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:11.248 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:11.247+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:11.251 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 112.310112535, 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:09:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.041917218999998, 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:09:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:09:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.040321470999999, 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:09:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:09:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.039510852000006, 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:09:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.074776897999996, 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:09:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.072120194999997, 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:09:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.489472652000003, 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:09:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:09:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.075159193000005, 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.573460171000001, 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.571883284000002, 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.542135694000002, 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:09:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.386756208000001, 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:09:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:09:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.383102778000001, 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:09:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.667436103, 2024-09-18T01:09:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:09:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:09:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 866 entries 100'101 (0'0) error 3:9153dd03:::rbd_data.117649d3c4d0.0000000000000093:head by client.4470.0:61161 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'99)", 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.821212930000002, 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048029769999999999, 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428633+0000", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7910000000000005e-06 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428653+0000", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109999999999999e-05 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428667+0000", 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4187e-05 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428764+0000", 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.433427+0000", 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046639020000000002 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 869 entries 100'103 (0'0) error 3:94e57bb7:::rbd_data.117649d3c4d0.00000000000000b2:head by client.4470.0:61192 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.807926152, 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047167169999999996, 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441912+0000", 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441920+0000", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2260000000000005e-06 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441942+0000", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1804e-05 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441957+0000", 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4599e-05 2024-09-18T01:09:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.442024+0000", 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7860000000000004e-05 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.446628+0000", 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046035479999999998 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 871 entries 100'104 (0'0) error 3:90b555ad:::rbd_data.117649d3c4d0.00000000000000ba:head by client.4470.0:61200 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.805399232999999, 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049806420000000004, 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444439+0000", 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444445+0000", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444459+0000", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4241e-05 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444468+0000", 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5399999999999996e-06 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444541+0000", 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.449419+0000", 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048779060000000004 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116214 3.4 e110/23 3:2420fb28:::rbd_data.11766701b08b.0000000000000277:head v 110'180, mlcod=108'178)", 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.19474988, 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045364760000000002, 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999997e-07 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055097+0000", 2024-09-18T01:09:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.208e-06 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055111+0000", 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545999999999999e-05 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055118+0000", 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.934e-06 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055175+0000", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7240000000000001e-05 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055201+0000", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5978000000000001e-05 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059539+0000", 2024-09-18T01:09:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043376889999999996 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059624+0000", 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5118999999999996e-05 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116558 3.9 e110/23 3:9119117f:::rbd_data.11766701b08b.0000000000000127:head v 110'143, mlcod=110'141)", 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.041931904, 2024-09-18T01:09:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005246901, 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207914+0000", 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9929999999999995e-06 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207925+0000", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0828e-05 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207934+0000", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0860000000000008e-06 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207976+0000", 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2274000000000002e-05 2024-09-18T01:09:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207991+0000", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4296e-05 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213113+0000", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051225649999999999 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213152+0000", 2024-09-18T01:09:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9116999999999998e-05 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116699 3.4 e110/23 3:27f7a2f6:::rbd_data.11766701b08b.00000000000001bf:head v 110'196, mlcod=110'193)", 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.983309509, 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042393159999999999, 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266529+0000", 2024-09-18T01:09:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266534+0000", 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5500000000000002e-06 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266545+0000", 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0926e-05 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266551+0000", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3049999999999998e-06 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266596+0000", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4864999999999998e-05 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266617+0000", 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0682e-05 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270721+0000", 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041039140000000002 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270767+0000", 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.634e-05 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.342 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:11.343 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:11.387 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:11.385+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:11.446 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:11.446 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:11.451 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:11.449+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:11.455 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:11.551 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:11.550+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:11.597 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:11.598 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:11.606 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:11.652 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:11.650+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:11.656 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085976097, 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085951112, 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085931333000001, 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085904931000002, 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085883435, 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085864139000002, 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085846305, 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085820242, 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085800887000001, 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085770643, 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085752048, 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085733488999999, 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085715435000001, 2024-09-18T01:09:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085697675999999, 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085671407, 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085652787000001, 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085634537000001, 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.085083162, 2024-09-18T01:09:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.083484175999999, 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.079718189000001, 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:11.823 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:11.824 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:11.825 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:11.822+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:11.833 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:11.833 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:11.833 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-18T01:09:11.833 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-18T01:09:11.925 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:11.923+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:11.939 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:11.939 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:11.939 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:11.939 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:11.948 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:11.999 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.001 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.001 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-18T01:09:12.043 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:12.041+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:12.052 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:12.052 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:12.052 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:12.052 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:12.052 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:12.062 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:12.126 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:12.124+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.500591239000002, 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:09:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:09:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 74.442403659999997, 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302908057, 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:09:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302885538000002, 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:09:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302863970000001, 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:09:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302843201000002, 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:09:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302824006000002, 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:09:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:09:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.30280522, 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:09:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:09:12.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302785911000001, 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:12.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:09:12.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302763521999999, 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.218 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302742202000001, 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:09:12.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:09:12.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302718608999999, 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:09:12.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:12.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302698693, 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:12.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:09:12.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302680261999999, 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:09:12.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302655997999999, 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:09:12.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302631624, 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:09:12.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:09:12.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302610483999999, 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:09:12.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302590402, 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302570255999999, 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:09:12.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.302550192999998, 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:09:12.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.263 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:12.265 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.265 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:12.266 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:12.263+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:12.266 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:12.267 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:12.267 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:12.267 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.267 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-18T01:09:12.365 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:12.364+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:12.410 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:12.410 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:12.411 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:12.411 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:12.419 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:12.437 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:12.438 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.439 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.440 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-18T01:09:12.484 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:12.482+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:12.563 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:12.563 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:12.563 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:12.563 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:12.563 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:12.566 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:12.564+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:12.571 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:12.572 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:12.572 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.573 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:12.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.574 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.574 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-18T01:09:12.654 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:12.664 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:09:12.667 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:12.666+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.653086625, 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.678 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.653044473999998, 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.653011046000003, 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652973545999998, 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652951969, 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652932828999994, 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:12.701 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.705 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652912438000001, 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:12.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652883037999999, 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:12.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652854247999997, 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:12.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:12.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:12.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:12.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652833552999994, 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:12.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652814069000001, 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:12.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:12.728 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652780157999999, 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:12.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.652744158999994, 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:12.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.568296187000001, 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:12.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.568285978000006, 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:12.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.067884751999998, 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:12.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:12.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.052398582999999, 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.753 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:12.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.044507863000007, 2024-09-18T01:09:12.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:12.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:09:12.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.018438860000003, 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:12.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:12.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:12.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:12.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.013270067999997, 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:12.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:12.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:12.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:12.773 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:12.775 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:12.775 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:12.777 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.777 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.777 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-18T01:09:12.777 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:12.772+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:12.781 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:12.781 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:12.799 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:12.800 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:12.816 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:12.832 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:12.850 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:12.851 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:12.867 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:12.875 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:12.873+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:12.885 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:12.885 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:12.902 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:12.902 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:12.918 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:12.919 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:12.919 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-18T01:09:12.922 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:12.923 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:12.932 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:13.005 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:13.003+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:13.076 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:13.075+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:13.077 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.077 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:13.091 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:09:13.092 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:09:13.095 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:09:13.095 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:09:13.177 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:13.175+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.240 INFO:teuthology.orchestra.run.smithi120.stdout:2568 writesame 0xa9299a6 thru 0xa92bc6c data_size 0x1(0x22c7 bytes) 2024-09-18T01:09:13.241 INFO:teuthology.orchestra.run.smithi120.stdout:2569 read 0x16d0917 thru 0x16d6667 (0x5d51 bytes) 2024-09-18T01:09:13.241 INFO:teuthology.orchestra.run.smithi120.stdout:2570 writesame 0x323dd26 thru 0x3248a28 data_size 0x1(0xad03 bytes) 2024-09-18T01:09:13.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:13.261 INFO:teuthology.orchestra.run.smithi120.stdout:2573 writesame 0x135bf33 thru 0x1364cf6 data_size 0x56(0x8dc4 bytes) 2024-09-18T01:09:13.266 INFO:teuthology.orchestra.run.smithi120.stdout:2574 write 0x7a80707 thru 0x7a84490 (0x3d8a bytes) 2024-09-18T01:09:13.270 INFO:teuthology.orchestra.run.smithi120.stdout:2575 writesame 0x5cc403d thru 0x5ccae20 data_size 0x1a(0x6de4 bytes) 2024-09-18T01:09:13.275 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.275 INFO:teuthology.orchestra.run.smithi120.stdout:2577 read 0x8c89b74 thru 0x8c9646c (0xc8f9 bytes) 2024-09-18T01:09:13.279 INFO:teuthology.orchestra.run.smithi120.stdout:2578 trunc from 0xa92bc6d to 0xdf5001e 2024-09-18T01:09:13.284 INFO:teuthology.orchestra.run.smithi120.stdout:2581 punch from 0x4b742c4 to 0x4b7af9c, (0x6cd8 bytes) 2024-09-18T01:09:13.285 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:13.284+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:13.287 INFO:teuthology.orchestra.run.smithi120.stdout:2582 trunc from 0xdf5001e to 0xc1ae8b4 2024-09-18T01:09:13.292 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.292 INFO:teuthology.orchestra.run.smithi120.stdout:2584 read 0x3d34da thru 0x3df1a0 (0xbcc7 bytes) 2024-09-18T01:09:13.293 INFO:teuthology.orchestra.run.smithi120.stdout:2585 read 0xa65d8e1 thru 0xa66a9ac (0xd0cc bytes) 2024-09-18T01:09:13.293 INFO:teuthology.orchestra.run.smithi120.stdout:2587 read 0x617ae34 thru 0x61807e6 (0x59b3 bytes) 2024-09-18T01:09:13.293 INFO:teuthology.orchestra.run.smithi120.stdout:2588 read 0x3fe478 thru 0x40cdab (0xe934 bytes) 2024-09-18T01:09:13.294 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.294 INFO:teuthology.orchestra.run.smithi120.stdout:2590 write 0x16f030a thru 0x16fa25e (0x9f55 bytes) 2024-09-18T01:09:13.298 INFO:teuthology.orchestra.run.smithi120.stdout:2592 write 0x7fd7e52 thru 0x7fde761 (0x6910 bytes) 2024-09-18T01:09:13.301 INFO:teuthology.orchestra.run.smithi120.stdout:2593 write 0xb79788b thru 0xb79b06d (0x37e3 bytes) 2024-09-18T01:09:13.304 INFO:teuthology.orchestra.run.smithi120.stdout:2594 read 0x25c0233 thru 0x25ca9ef (0xa7bd bytes) 2024-09-18T01:09:13.304 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.304 INFO:teuthology.orchestra.run.smithi120.stdout:2597 write 0xb8be9d9 thru 0xb8ccc1a (0xe242 bytes) 2024-09-18T01:09:13.308 INFO:teuthology.orchestra.run.smithi120.stdout:2598 read 0xb041179 thru 0xb04d509 (0xc391 bytes) 2024-09-18T01:09:13.308 INFO:teuthology.orchestra.run.smithi120.stdout:2599 write 0x1efd2f0 thru 0x1f0775b (0xa46c bytes) 2024-09-18T01:09:13.310 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.310 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.310 INFO:teuthology.orchestra.run.smithi120.stdout:2602 punch from 0x2b4fa41 to 0x2b519a6, (0x1f65 bytes) 2024-09-18T01:09:13.310 INFO:teuthology.orchestra.run.smithi120.stdout:2603 writesame 0x87687f3 thru 0x877139c data_size 0x76(0x8baa bytes) 2024-09-18T01:09:13.317 INFO:teuthology.orchestra.run.smithi120.stdout:2604 writesame 0x118ceb thru 0x1234cd data_size 0x1(0xa7e3 bytes) 2024-09-18T01:09:13.335 INFO:teuthology.orchestra.run.smithi120.stdout:2605 read 0xb485c6e thru 0xb494232 (0xe5c5 bytes) 2024-09-18T01:09:13.335 INFO:teuthology.orchestra.run.smithi120.stdout:2606 write 0x40da1b2 thru 0x40e3141 (0x8f90 bytes) 2024-09-18T01:09:13.352 INFO:teuthology.orchestra.run.smithi120.stdout:2607 trunc from 0xc1ae8b4 to 0x11a888b 2024-09-18T01:09:13.353 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.353 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:13.373 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.386 INFO:teuthology.orchestra.run.smithi120.stdout:2609 write 0x7e4a549 thru 0x7e51a6a (0x7522 bytes) 2024-09-18T01:09:13.390 INFO:teuthology.orchestra.run.smithi120.stdout:2612 write 0x381d657 thru 0x3823aeb (0x6495 bytes) 2024-09-18T01:09:13.396 INFO:teuthology.orchestra.run.smithi120.stdout:2613 trunc from 0x7e51a6b to 0xa2d5398 2024-09-18T01:09:13.403 INFO:teuthology.orchestra.run.smithi120.stdout:2616 read 0x8272810 thru 0x82763d9 (0x3bca bytes) 2024-09-18T01:09:13.403 INFO:teuthology.orchestra.run.smithi120.stdout:2617 punch from 0x78f1132 to 0x78f368c, (0x255a bytes) 2024-09-18T01:09:13.403 INFO:teuthology.orchestra.run.smithi120.stdout:2618 read 0x10dca73 thru 0x10e2f1e (0x64ac bytes) 2024-09-18T01:09:13.403 INFO:teuthology.orchestra.run.smithi120.stdout:2619 read 0x755a472 thru 0x7561146 (0x6cd5 bytes) 2024-09-18T01:09:13.403 INFO:teuthology.orchestra.run.smithi120.stdout:2621 read 0x9269ab9 thru 0x9273e52 (0xa39a bytes) 2024-09-18T01:09:13.405 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:13.404+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:13.413 INFO:teuthology.orchestra.run.smithi120.stdout:2622 writesame 0xdac4327 thru 0xdac9dd5 data_size 0x1(0x5aaf bytes) 2024-09-18T01:09:13.414 INFO:teuthology.orchestra.run.smithi120.stdout:2623 trunc from 0xdac9dd6 to 0xd1e9702 2024-09-18T01:09:13.419 INFO:teuthology.orchestra.run.smithi120.stdout:2624 read 0x5393057 thru 0x5395d24 (0x2cce bytes) 2024-09-18T01:09:13.419 INFO:teuthology.orchestra.run.smithi120.stdout:2625 writesame 0xd145840 thru 0xd14fe94 data_size 0x4d(0xa655 bytes) 2024-09-18T01:09:13.423 INFO:teuthology.orchestra.run.smithi120.stdout:2626 read 0xc71c464 thru 0xc724297 (0x7e34 bytes) 2024-09-18T01:09:13.423 INFO:teuthology.orchestra.run.smithi120.stdout:2627 writesame 0x87f5577 thru 0x87f6f2c data_size 0x3(0x19b6 bytes) 2024-09-18T01:09:13.429 INFO:teuthology.orchestra.run.smithi120.stdout:2628 read 0x4375d0b thru 0x437e72e (0x8a24 bytes) 2024-09-18T01:09:13.437 INFO:teuthology.orchestra.run.smithi120.stdout:2629 trunc from 0xd1e9702 to 0x4a1014c 2024-09-18T01:09:13.450 INFO:teuthology.orchestra.run.smithi120.stdout:2630 punch from 0x653d7a to 0x662b71, (0xedf7 bytes) 2024-09-18T01:09:13.450 INFO:teuthology.orchestra.run.smithi120.stdout:2632 read 0xfe1636 thru 0xfec2d6 (0xaca1 bytes) 2024-09-18T01:09:13.450 INFO:teuthology.orchestra.run.smithi120.stdout:2634 write 0x611de thru 0x6a567 (0x938a bytes) 2024-09-18T01:09:13.452 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.452 INFO:teuthology.orchestra.run.smithi120.stdout:2637 punch from 0xfba582 to 0xfc6c8b, (0xc709 bytes) 2024-09-18T01:09:13.460 INFO:teuthology.orchestra.run.smithi120.stdout:2638 write 0x65b9f28 thru 0x65c6865 (0xc93e bytes) 2024-09-18T01:09:13.465 INFO:teuthology.orchestra.run.smithi120.stdout:2639 write 0x1645b08 thru 0x164c817 (0x6d10 bytes) 2024-09-18T01:09:13.469 INFO:teuthology.orchestra.run.smithi120.stdout:2640 punch from 0x5cd7209 to 0x5ce3198, (0xbf8f bytes) 2024-09-18T01:09:13.469 INFO:teuthology.orchestra.run.smithi120.stdout:2641 read 0x3f691bc thru 0x3f71642 (0x8487 bytes) 2024-09-18T01:09:13.476 INFO:teuthology.orchestra.run.smithi120.stdout:2644 writesame 0x8269f2c thru 0x8275c88 data_size 0x71(0xbd5d bytes) 2024-09-18T01:09:13.476 INFO:teuthology.orchestra.run.smithi120.stdout:2645 punch from 0x3901b13 to 0x3910a2a, (0xef17 bytes) 2024-09-18T01:09:13.477 INFO:teuthology.orchestra.run.smithi120.stdout:2647 writesame 0xa65231 thru 0xa70738 data_size 0xc(0xb508 bytes) 2024-09-18T01:09:13.478 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.478 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.480 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:13.478+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:13.491 INFO:teuthology.orchestra.run.smithi120.stdout:2650 write 0xd9b89c5 thru 0xd9bf573 (0x6baf bytes) 2024-09-18T01:09:13.494 INFO:teuthology.orchestra.run.smithi120.stdout:2651 read 0x253a80a thru 0x254a0e3 (0xf8da bytes) 2024-09-18T01:09:13.494 INFO:teuthology.orchestra.run.smithi120.stdout:2652 write 0x7288203 thru 0x729503e (0xce3c bytes) 2024-09-18T01:09:13.498 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.498 INFO:teuthology.orchestra.run.smithi120.stdout:2655 write 0x7037923 thru 0x703bc2b (0x4309 bytes) 2024-09-18T01:09:13.500 INFO:teuthology.orchestra.run.smithi120.stdout:2656 read 0x56d949b thru 0x56de64d (0x51b3 bytes) 2024-09-18T01:09:13.500 INFO:teuthology.orchestra.run.smithi120.stdout:2657 writesame 0x5a8f44e thru 0x5a9696a data_size 0x1(0x751d bytes) 2024-09-18T01:09:13.515 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.515 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.515 INFO:teuthology.orchestra.run.smithi120.stdout:2659 read 0x53969a6 thru 0x539c19b (0x57f6 bytes) 2024-09-18T01:09:13.516 INFO:teuthology.orchestra.run.smithi120.stdout:2660 read 0x56081bc thru 0x561141b (0x9260 bytes) 2024-09-18T01:09:13.516 INFO:teuthology.orchestra.run.smithi120.stdout:2661 punch from 0x1f20bc4 to 0x1f2228e, (0x16ca bytes) 2024-09-18T01:09:13.516 INFO:teuthology.orchestra.run.smithi120.stdout:2662 write 0x4391617 thru 0x4397b21 (0x650b bytes) 2024-09-18T01:09:13.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:13.525 INFO:teuthology.orchestra.run.smithi120.stdout:2663 writesame 0xe27dfd2 thru 0xe284b3e data_size 0x3(0x6b6d bytes) 2024-09-18T01:09:13.529 INFO:teuthology.orchestra.run.smithi120.stdout:2665 trunc from 0xe284b3f to 0x9d047a2 2024-09-18T01:09:13.539 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:2669 writesame 0x47b4667 thru 0x47c3cbb data_size 0x1(0xf655 bytes) 2024-09-18T01:09:13.563 INFO:teuthology.orchestra.run.smithi120.stdout:2671 write 0x3b9e2c4 thru 0x3bad92d (0xf66a bytes) 2024-09-18T01:09:13.567 INFO:teuthology.orchestra.run.smithi120.stdout:2672 read 0xe59184 thru 0xe5bb71 (0x29ee bytes) 2024-09-18T01:09:13.568 INFO:teuthology.orchestra.run.smithi120.stdout:2674 write 0x1e4c58c thru 0x1e58811 (0xc286 bytes) 2024-09-18T01:09:13.580 INFO:teuthology.orchestra.run.smithi120.stdout:2676 write 0xb18527b thru 0xb189217 (0x3f9d bytes) 2024-09-18T01:09:13.580 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:13.578+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:13.583 INFO:teuthology.orchestra.run.smithi120.stdout:2677 writesame 0x8a0d0cd thru 0x8a162af data_size 0x3b(0x91e3 bytes) 2024-09-18T01:09:13.588 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.588 INFO:teuthology.orchestra.run.smithi120.stdout:2681 punch from 0x47b7c62 to 0x47c5371, (0xd70f bytes) 2024-09-18T01:09:13.588 INFO:teuthology.orchestra.run.smithi120.stdout:2684 punch from 0x6eff0ee to 0x6f0ef81, (0xfe93 bytes) 2024-09-18T01:09:13.588 INFO:teuthology.orchestra.run.smithi120.stdout:2685 read 0x8d8a9a7 thru 0x8d952f5 (0xa94f bytes) 2024-09-18T01:09:13.591 INFO:teuthology.orchestra.run.smithi120.stdout:2686 trunc from 0xb189218 to 0xd868b74 2024-09-18T01:09:13.597 INFO:teuthology.orchestra.run.smithi120.stdout:2687 write 0xba3fdd8 thru 0xba4be3a (0xc063 bytes) 2024-09-18T01:09:13.604 INFO:teuthology.orchestra.run.smithi120.stdout:2688 read 0x658d8c7 thru 0x6593ecd (0x6607 bytes) 2024-09-18T01:09:13.605 INFO:teuthology.orchestra.run.smithi120.stdout:2689 read 0x4ef0bf4 thru 0x4ef882d (0x7c3a bytes) 2024-09-18T01:09:13.605 INFO:teuthology.orchestra.run.smithi120.stdout:2691 punch from 0x5d63448 to 0x5d6a186, (0x6d3e bytes) 2024-09-18T01:09:13.605 INFO:teuthology.orchestra.run.smithi120.stdout:2692 writesame 0xbd96cfb thru 0xbd9aa56 data_size 0x44(0x3d5c bytes) 2024-09-18T01:09:13.623 INFO:teuthology.orchestra.run.smithi120.stdout:2694 trunc from 0xd868b74 to 0x501ede 2024-09-18T01:09:13.658 INFO:teuthology.orchestra.run.smithi120.stdout:2695 write 0x48ce98c thru 0x48dbef2 (0xd567 bytes) 2024-09-18T01:09:13.665 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.665 INFO:teuthology.orchestra.run.smithi120.stdout:2697 punch from 0x428e98b to 0x4294b29, (0x619e bytes) 2024-09-18T01:09:13.675 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.675 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.676 INFO:teuthology.orchestra.run.smithi120.stdout:2698 writesame 0x8c32792 thru 0x8c35d94 data_size 0xb(0x3603 bytes) 2024-09-18T01:09:13.676 INFO:teuthology.orchestra.run.smithi120.stdout:2699 read 0x40fca97 thru 0x4109aea (0xd054 bytes) 2024-09-18T01:09:13.676 INFO:teuthology.orchestra.run.smithi120.stdout:2701 read 0x68b440b thru 0x68b78a3 (0x3499 bytes) 2024-09-18T01:09:13.676 INFO:teuthology.orchestra.run.smithi120.stdout:2702 write 0x2913a83 thru 0x2915764 (0x1ce2 bytes) 2024-09-18T01:09:13.679 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.679 INFO:teuthology.orchestra.run.smithi120.stdout:2705 write 0x6c19405 thru 0x6c1e3ac (0x4fa8 bytes) 2024-09-18T01:09:13.681 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:13.680+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:13.684 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.684 INFO:teuthology.orchestra.run.smithi120.stdout:2709 write 0x2e3ef54 thru 0x2e4b65e (0xc70b bytes) 2024-09-18T01:09:13.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:13.697 INFO:teuthology.orchestra.run.smithi120.stdout:2710 write 0x9f3c407 thru 0x9f406ae (0x42a8 bytes) 2024-09-18T01:09:13.711 INFO:teuthology.orchestra.run.smithi120.stdout:2711 write 0xe3da730 thru 0xe3de1c1 (0x3a92 bytes) 2024-09-18T01:09:13.715 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.715 INFO:teuthology.orchestra.run.smithi120.stdout:2713 write 0xad5fd74 thru 0xad633e4 (0x3671 bytes) 2024-09-18T01:09:13.719 INFO:teuthology.orchestra.run.smithi120.stdout:2714 read 0x8b222d7 thru 0x8b2bf62 (0x9c8c bytes) 2024-09-18T01:09:13.719 INFO:teuthology.orchestra.run.smithi120.stdout:2715 punch from 0xa2d2d9 to 0xa32567, (0x528e bytes) 2024-09-18T01:09:13.719 INFO:teuthology.orchestra.run.smithi120.stdout:2717 writesame 0xb3ba7ec thru 0xb3baa38 data_size 0x1(0x24d bytes) 2024-09-18T01:09:13.722 INFO:teuthology.orchestra.run.smithi120.stdout:2719 punch from 0xad60e9a to 0xad6c903, (0xba69 bytes) 2024-09-18T01:09:13.722 INFO:teuthology.orchestra.run.smithi120.stdout:2720 punch from 0x8999b6e to 0x89a61ce, (0xc660 bytes) 2024-09-18T01:09:13.722 INFO:teuthology.orchestra.run.smithi120.stdout:2721 write 0x3b65d8 thru 0x3b70eb (0xb14 bytes) 2024-09-18T01:09:13.725 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.725 INFO:teuthology.orchestra.run.smithi120.stdout:2723 read 0x993420e thru 0x9939810 (0x5603 bytes) 2024-09-18T01:09:13.725 INFO:teuthology.orchestra.run.smithi120.stdout:2725 write 0xcfab26e thru 0xcfb06f0 (0x5483 bytes) 2024-09-18T01:09:13.728 INFO:teuthology.orchestra.run.smithi120.stdout:2726 writesame 0x8af70ca thru 0x8afd579 data_size 0x48(0x64b0 bytes) 2024-09-18T01:09:13.731 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.745 INFO:teuthology.orchestra.run.smithi120.stdout:2728 trunc from 0xe3de1c2 to 0x20ddfa9 2024-09-18T01:09:13.760 INFO:teuthology.orchestra.run.smithi120.stdout:2729 read 0x1469f36 thru 0x146c5f9 (0x26c4 bytes) 2024-09-18T01:09:13.760 INFO:teuthology.orchestra.run.smithi120.stdout:2732 writesame 0xf7afd9 thru 0xf7d214 data_size 0xe(0x223c bytes) 2024-09-18T01:09:13.762 INFO:teuthology.orchestra.run.smithi120.stdout:2733 trunc from 0x20ddfa9 to 0x1d0b1f5 2024-09-18T01:09:13.781 INFO:teuthology.orchestra.run.smithi120.stdout:2734 write 0x21895a4 thru 0x219667b (0xd0d8 bytes) 2024-09-18T01:09:13.782 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:13.780+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:13.807 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.807 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.807 INFO:teuthology.orchestra.run.smithi120.stdout:2735 write 0xdeab2ce thru 0xdeb36db (0x840e bytes) 2024-09-18T01:09:13.811 INFO:teuthology.orchestra.run.smithi120.stdout:2736 punch from 0xb29a19f to 0xb29dbff, (0x3a60 bytes) 2024-09-18T01:09:13.811 INFO:teuthology.orchestra.run.smithi120.stdout:2742 read 0x67dcd74 thru 0x67e8df5 (0xc082 bytes) 2024-09-18T01:09:13.811 INFO:teuthology.orchestra.run.smithi120.stdout:2743 punch from 0x2bca2b9 to 0x2bcb9df, (0x1726 bytes) 2024-09-18T01:09:13.811 INFO:teuthology.orchestra.run.smithi120.stdout:2745 write 0xb6b6d83 thru 0xb6c2f11 (0xc18f bytes) 2024-09-18T01:09:13.814 INFO:teuthology.orchestra.run.smithi120.stdout:2748 read 0x5f4062e thru 0x5f41d75 (0x1748 bytes) 2024-09-18T01:09:13.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:13.828 INFO:teuthology.orchestra.run.smithi120.stdout:2749 trunc from 0xdeb36dc to 0x2ee8cd8 2024-09-18T01:09:13.839 INFO:teuthology.orchestra.run.smithi120.stdout:2750 read 0x2a3bb00 thru 0x2a3d544 (0x1a45 bytes) 2024-09-18T01:09:13.848 INFO:teuthology.orchestra.run.smithi120.stdout:2751 trunc from 0x2ee8cd8 to 0x9d1fe10 2024-09-18T01:09:13.854 INFO:teuthology.orchestra.run.smithi120.stdout:2753 read 0x49ce454 thru 0x49dcd51 (0xe8fe bytes) 2024-09-18T01:09:13.854 INFO:teuthology.orchestra.run.smithi120.stdout:2754 read 0x4281c6f thru 0x4290f72 (0xf304 bytes) 2024-09-18T01:09:13.854 INFO:teuthology.orchestra.run.smithi120.stdout:2756 read 0x2a6ed11 thru 0x2a7a8be (0xbbae bytes) 2024-09-18T01:09:13.854 INFO:teuthology.orchestra.run.smithi120.stdout:2757 read 0x58af1ff thru 0x58bcf7c (0xdd7e bytes) 2024-09-18T01:09:13.854 INFO:teuthology.orchestra.run.smithi120.stdout:2758 punch from 0x7129b18 to 0x713924e, (0xf736 bytes) 2024-09-18T01:09:13.855 INFO:teuthology.orchestra.run.smithi120.stdout:2759 read 0x270dd7a thru 0x27113e2 (0x3669 bytes) 2024-09-18T01:09:13.855 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.855 INFO:teuthology.orchestra.run.smithi120.stdout:2761 write 0x71b7955 thru 0x71c44bd (0xcb69 bytes) 2024-09-18T01:09:13.858 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.861 INFO:teuthology.orchestra.run.smithi120.stdout:2763 trunc from 0x9d1fe10 to 0x72a88ef 2024-09-18T01:09:13.874 INFO:teuthology.orchestra.run.smithi120.stdout:2765 trunc from 0x72a88ef to 0x262a958 2024-09-18T01:09:13.883 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:13.882+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:13.892 INFO:teuthology.orchestra.run.smithi120.stdout:2766 read 0x1cbbdb7 thru 0x1cc3c86 (0x7ed0 bytes) 2024-09-18T01:09:13.892 INFO:teuthology.orchestra.run.smithi120.stdout:2768 punch from 0x11e215d to 0x11f1e3f, (0xfce2 bytes) 2024-09-18T01:09:13.895 INFO:teuthology.orchestra.run.smithi120.stdout:2769 trunc from 0x262a958 to 0x4f1d490 2024-09-18T01:09:13.901 INFO:teuthology.orchestra.run.smithi120.stdout:2770 punch from 0x26f6347 to 0x26fc8c3, (0x657c bytes) 2024-09-18T01:09:13.901 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.917 INFO:teuthology.orchestra.run.smithi120.stdout:2772 write 0xdd6409f thru 0xdd66c96 (0x2bf8 bytes) 2024-09-18T01:09:13.921 INFO:teuthology.orchestra.run.smithi120.stdout:2774 punch from 0x8e926b9 to 0x8e93b6b, (0x14b2 bytes) 2024-09-18T01:09:13.921 INFO:teuthology.orchestra.run.smithi120.stdout:2775 read 0xd2c7deb thru 0xd2d56b9 (0xd8cf bytes) 2024-09-18T01:09:13.921 INFO:teuthology.orchestra.run.smithi120.stdout:2777 writesame 0xc5e2528 thru 0xc5e4642 data_size 0xf(0x211b bytes) 2024-09-18T01:09:13.922 INFO:teuthology.orchestra.run.smithi120.stdout:2778 punch from 0xcea8c34 to 0xceb5a0f, (0xcddb bytes) 2024-09-18T01:09:13.922 INFO:teuthology.orchestra.run.smithi120.stdout:2779 write 0x79c4874 thru 0x79c5570 (0xcfd bytes) 2024-09-18T01:09:13.936 INFO:teuthology.orchestra.run.smithi120.stdout:2780 trunc from 0xdd66c97 to 0x48d5b6e 2024-09-18T01:09:13.945 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:13.945 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:13.948 INFO:teuthology.orchestra.run.smithi120.stdout:2782 read 0x1cdb1fe thru 0x1ce1c09 (0x6a0c bytes) 2024-09-18T01:09:13.949 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.949 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:13.950 INFO:teuthology.orchestra.run.smithi120.stdout:2786 trunc from 0x48d5b6e to 0x35cca90 2024-09-18T01:09:13.956 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:13.967 INFO:teuthology.orchestra.run.smithi120.stdout:2787 write 0x7322fa7 thru 0x7323b17 (0xb71 bytes) 2024-09-18T01:09:13.979 INFO:teuthology.orchestra.run.smithi120.stdout:2788 write 0x88c4db0 thru 0x88c99ff (0x4c50 bytes) 2024-09-18T01:09:13.984 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:13.982+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:13.991 INFO:teuthology.orchestra.run.smithi120.stdout:2789 writesame 0xa4458b5 thru 0xa449338 data_size 0x46(0x3a84 bytes) 2024-09-18T01:09:13.991 INFO:teuthology.orchestra.run.smithi120.stdout:2793 punch from 0x839ccd to 0x83b069, (0x139c bytes) 2024-09-18T01:09:13.991 INFO:teuthology.orchestra.run.smithi120.stdout:2794 punch from 0x6ab723b to 0x6ac0f52, (0x9d17 bytes) 2024-09-18T01:09:14.001 INFO:teuthology.orchestra.run.smithi120.stdout:2795 writesame 0xcebdea7 thru 0xcec8e2f data_size 0x3(0xaf89 bytes) 2024-09-18T01:09:14.007 INFO:teuthology.orchestra.run.smithi120.stdout:2797 write 0xd29349d thru 0xd2a0ffc (0xdb60 bytes) 2024-09-18T01:09:14.013 INFO:teuthology.orchestra.run.smithi120.stdout:2798 read 0xb6235ca thru 0xb627cf3 (0x472a bytes) 2024-09-18T01:09:14.013 INFO:teuthology.orchestra.run.smithi120.stdout:2799 punch from 0x4a0210c to 0x4a074c6, (0x53ba bytes) 2024-09-18T01:09:14.016 INFO:teuthology.orchestra.run.smithi120.stdout:2800 trunc from 0xd2a0ffd to 0xa16b1b6 2024-09-18T01:09:14.029 INFO:teuthology.orchestra.run.smithi120.stdout:2801 read 0x884fdf6 thru 0x88551c7 (0x53d2 bytes) 2024-09-18T01:09:14.042 INFO:teuthology.orchestra.run.smithi120.stdout:2802 write 0xe95c025 thru 0xe96847f (0xc45b bytes) 2024-09-18T01:09:14.047 INFO:teuthology.orchestra.run.smithi120.stdout:2803 punch from 0x47e41db to 0x47ebec5, (0x7cea bytes) 2024-09-18T01:09:14.047 INFO:teuthology.orchestra.run.smithi120.stdout:2806 read 0x7f083ee thru 0x7f08915 (0x528 bytes) 2024-09-18T01:09:14.047 INFO:teuthology.orchestra.run.smithi120.stdout:2809 write 0x97c5ee1 thru 0x97c85a2 (0x26c2 bytes) 2024-09-18T01:09:14.049 INFO:teuthology.orchestra.run.smithi120.stdout:2811 write 0x846f4e6 thru 0x846fcae (0x7c9 bytes) 2024-09-18T01:09:14.052 INFO:teuthology.orchestra.run.smithi120.stdout:2812 write 0x33245e5 thru 0x332c71d (0x8139 bytes) 2024-09-18T01:09:14.055 INFO:teuthology.orchestra.run.smithi120.stdout:2813 write 0x1aa10c9 thru 0x1aad33d (0xc275 bytes) 2024-09-18T01:09:14.057 INFO:teuthology.orchestra.run.smithi120.stdout:2814 write 0x6addd9e thru 0x6ae59ae (0x7c11 bytes) 2024-09-18T01:09:14.079 INFO:teuthology.orchestra.run.smithi120.stdout:2815 trunc from 0xe968480 to 0x29cdb38 2024-09-18T01:09:14.084 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:14.083+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:14.086 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.087 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.097 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:14.113 INFO:teuthology.orchestra.run.smithi120.stdout:2817 write 0x6b0375b thru 0x6b110ea (0xd990 bytes) 2024-09-18T01:09:14.134 INFO:teuthology.orchestra.run.smithi120.stdout:2819 write 0xcc66c46 thru 0xcc6c293 (0x564e bytes) 2024-09-18T01:09:14.138 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:14.138 INFO:teuthology.orchestra.run.smithi120.stdout:2822 read 0x674e4b0 thru 0x6754a37 (0x6588 bytes) 2024-09-18T01:09:14.138 INFO:teuthology.orchestra.run.smithi120.stdout:2823 write 0x1950d40 thru 0x195e5fc (0xd8bd bytes) 2024-09-18T01:09:14.139 INFO:teuthology.orchestra.run.smithi120.stdout:2824 punch from 0x2ffa13 to 0x305b7e, (0x616b bytes) 2024-09-18T01:09:14.141 INFO:teuthology.orchestra.run.smithi120.stdout:2825 trunc from 0xcc6c294 to 0xbcf2b5e 2024-09-18T01:09:14.150 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:14.159 INFO:teuthology.orchestra.run.smithi120.stdout:2829 writesame 0xe5b623b thru 0xe5c36db data_size 0x1(0xd4a1 bytes) 2024-09-18T01:09:14.159 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:14.164 INFO:teuthology.orchestra.run.smithi120.stdout:2831 trunc from 0xe5c36dc to 0x9881fb5 2024-09-18T01:09:14.170 INFO:teuthology.orchestra.run.smithi120.stdout:2832 punch from 0x7858af6 to 0x78610ba, (0x85c4 bytes) 2024-09-18T01:09:14.170 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:14.170 INFO:teuthology.orchestra.run.smithi120.stdout:2835 write 0x6766664 thru 0x676b710 (0x50ad bytes) 2024-09-18T01:09:14.173 INFO:teuthology.orchestra.run.smithi120.stdout:2836 write 0x9406ba5 thru 0x940b27b (0x46d7 bytes) 2024-09-18T01:09:14.176 INFO:teuthology.orchestra.run.smithi120.stdout:2837 clone 27 order 18 su 65536 sc 3 2024-09-18T01:09:14.185 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:14.184+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:14.221 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.222 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:14.285 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:14.284+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:14.372 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.372 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.381 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:14.425 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:14.423+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:14.484 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.484 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.488 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:14.487+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:14.492 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:14.589 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:14.588+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:14.641 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.641 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:14.690 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:14.689+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:14.753 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.753 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.762 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:14.775 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:09:14.791 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:14.790+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:14.868 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.868 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:14.913 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:14.921 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:14.919+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:14.987 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:14.987 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:14.992 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:14.990+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:14.997 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:15.044 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:15.044 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:15.063 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:15.063 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:15.079 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:15.079 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:15.093 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:15.091+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:15.097 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:15.097 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:15.114 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:15.114 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:15.131 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:15.143 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:15.143 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:15.143 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:15.148 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:15.148 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:15.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:15.166 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:15.166 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:15.219 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:15.217+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:15.269 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:15.269 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:15.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639770291000001, 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.270 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:15.274 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639745306000002, 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:15.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639725527, 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639699125, 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.283 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639677629000001, 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:15.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.291 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639658333, 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:15.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639640498999999, 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.297 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639614435999999, 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:15.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:15.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639595081, 2024-09-18T01:09:15.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.309 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639564837000002, 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:15.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:15.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:15.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.313 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639546242000002, 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639527683000001, 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.318 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:15.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:15.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639509628999999, 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:15.322 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.326 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639491870000001, 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:15.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639465601000001, 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.331 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639446980999999, 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:15.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:15.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.639428730999999, 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:15.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:15.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.638877355999998, 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.637278370000001, 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:15.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:15.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:15.352 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.633512382999999, 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:15.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:15.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:15.357 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:15.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:15.359 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:15.357+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:15.459 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:15.458+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:15.474 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:15.474 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:15.483 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:15.556 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:09:15.560 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:15.559+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:15.593 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:15.594 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:15.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:15.688 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:15.687+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:15.763 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:15.760+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:15.773 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.105412357999995, 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:09:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:09:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:09:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:09:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.047224779000004, 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907729176, 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:09:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907706656999999, 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:09:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:09:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907685089000001, 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:09:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907664319999999, 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:09:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907645124999998, 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:09:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:09:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907626339, 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.827 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907607030000001, 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:09:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907584641, 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:09:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907563321000001, 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:09:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907539728, 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:09:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907519812, 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:09:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907501380999999, 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907477116999999, 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:09:15.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907452743, 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907431602999999, 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:15.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907411521, 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:09:15.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:15.881 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907391375, 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:09:15.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.907371311999999, 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:15.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:09:15.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:15.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:15.891 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:15.892 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:15.893 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:15.890+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:15.992 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:15.990+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:16.036 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.036 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:16.036 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:16.036 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.045 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:16.093 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:16.091+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:16.148 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:16.149 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.157 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:16.194 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:16.193+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:16.262 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.262 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.23713316, 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:16.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.237091009, 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:16.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:16.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.237057581, 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:16.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:16.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:16.278 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.237020081, 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:16.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236998504, 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:16.283 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:16.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:16.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.23697936400001, 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:16.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:16.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:16.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236958973, 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236929573, 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:16.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236900783, 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.305 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:16.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.309 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.23688008800001, 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:16.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236860604, 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:16.314 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:16.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.236826693, 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:16.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:16.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.23679069400001, 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:16.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:16.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.152342722, 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:16.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:16.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.152332513, 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:16.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.651931286999996, 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:16.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:16.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:16.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:16.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.636445117999997, 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:16.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:16.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:09:16.347 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.628554398000006, 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:09:16.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.602485395000002, 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:16.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:16.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.597316602999996, 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:16.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:16.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:16.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.362 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:16.364 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:16.362+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:16.465 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:16.464+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.477 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:16.566 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:16.564+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:16.623 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.623 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:16.623 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:16.624 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:16.624 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.632 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:16.667 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:16.665+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:16.692 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-18T01:09:16.696 INFO:teuthology.orchestra.run.smithi120.stdout:2838 writesame 0x7a5bde3 thru 0x7a69eea data_size 0x4c(0xe108 bytes) 2024-09-18T01:09:16.704 INFO:teuthology.orchestra.run.smithi120.stdout:2839 punch from 0x8b605d6 to 0x8b620c7, (0x1af1 bytes) 2024-09-18T01:09:16.704 INFO:teuthology.orchestra.run.smithi120.stdout:2840 punch from 0x8be1ac0 to 0x8be57f9, (0x3d39 bytes) 2024-09-18T01:09:16.713 INFO:teuthology.orchestra.run.smithi120.stdout:2841 write 0xb448bb7 thru 0xb45161a (0x8a64 bytes) 2024-09-18T01:09:16.721 INFO:teuthology.orchestra.run.smithi120.stdout:2843 trunc from 0xb45161b to 0xdcf5667 2024-09-18T01:09:16.727 INFO:teuthology.orchestra.run.smithi120.stdout:2844 read 0x1fa1bc5 thru 0x1fa2497 (0x8d3 bytes) 2024-09-18T01:09:16.728 INFO:teuthology.orchestra.run.smithi120.stdout:2845 writesame 0x4c8b70d thru 0x4c978ce data_size 0x15(0xc1c2 bytes) 2024-09-18T01:09:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:2846 read 0x745a1bd thru 0x7466175 (0xbfb9 bytes) 2024-09-18T01:09:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.733 INFO:teuthology.orchestra.run.smithi120.stdout:2850 write 0x981379d thru 0x9814e82 (0x16e6 bytes) 2024-09-18T01:09:16.735 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.746 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.173774842, 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:16.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:16.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.121783698999998, 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:16.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:16.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.121706903000003, 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:16.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:16.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.11720175, 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:16.763 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:16.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:16.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.116945954000002, 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.112920625999998, 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.112779476, 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:16.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:16.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:16.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.112546571999999, 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:16.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:16.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:16.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.109927509999999, 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.788 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:16.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:16.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:16.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.087588476000001, 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:16.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:16.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:16.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:16.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.137073202, 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:16.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.137051460999999, 2024-09-18T01:09:16.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:16.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:16.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.137031053000001, 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:16.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:16.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:16.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:16.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:16.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.137011394000002, 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:16.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136978059, 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.818 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136943527, 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:16.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:16.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136898637000002, 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:16.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136872262000001, 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.832 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:16.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.836 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136853111000001, 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:16.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:16.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.136834542999999, 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:16.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:16.846 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:16.846 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:16.848 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:16.848 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:16.846+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:16.848 INFO:teuthology.orchestra.run.smithi120.stdout:2852 trunc from 0xdcf5667 to 0x2d8d30d 2024-09-18T01:09:16.848 INFO:teuthology.orchestra.run.smithi120.stdout:2853 trunc from 0x2d8d30d to 0x16bc821 2024-09-18T01:09:16.848 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi120.stdout:2855 read 0x13e0f0c thru 0x13ee9f7 (0xdaec bytes) 2024-09-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi120.stdout:2856 read 0x64c6ab thru 0x64e0ae (0x1a04 bytes) 2024-09-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi120.stdout:2858 punch from 0xa2fab to 0xaf9cb, (0xca20 bytes) 2024-09-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi120.stdout:2859 punch from 0x151054d to 0x1514268, (0x3d1b bytes) 2024-09-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi120.stdout:2860 write 0xe44f396 thru 0xe45eab9 (0xf724 bytes) 2024-09-18T01:09:16.857 INFO:teuthology.orchestra.run.smithi120.stdout:2861 trunc from 0xe45eaba to 0x6618074 2024-09-18T01:09:16.870 INFO:teuthology.orchestra.run.smithi120.stdout:2863 read 0x2da2fa2 thru 0x2db0fe0 (0xe03f bytes) 2024-09-18T01:09:16.870 INFO:teuthology.orchestra.run.smithi120.stdout:2864 writesame 0x57659c9 thru 0x576a92d data_size 0x19(0x4f65 bytes) 2024-09-18T01:09:16.873 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.885 INFO:teuthology.orchestra.run.smithi120.stdout:2866 write 0x9b1c33b thru 0x9b22840 (0x6506 bytes) 2024-09-18T01:09:16.889 INFO:teuthology.orchestra.run.smithi120.stdout:2869 write 0x814e56d thru 0x814e674 (0x108 bytes) 2024-09-18T01:09:16.899 INFO:teuthology.orchestra.run.smithi120.stdout:2870 trunc from 0x9b22841 to 0x685bc16 2024-09-18T01:09:16.913 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.914 INFO:teuthology.orchestra.run.smithi120.stdout:2873 writesame 0x472a840 thru 0x4731919 data_size 0x6b(0x70da bytes) 2024-09-18T01:09:16.918 INFO:teuthology.orchestra.run.smithi120.stdout:2874 punch from 0x5d7b54e to 0x5d8a7e8, (0xf29a bytes) 2024-09-18T01:09:16.918 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.924 INFO:teuthology.orchestra.run.smithi120.stdout:2876 trunc from 0x685bc16 to 0x1c2392a 2024-09-18T01:09:16.935 INFO:teuthology.orchestra.run.smithi120.stdout:2877 read 0x117972e thru 0x117cf23 (0x37f6 bytes) 2024-09-18T01:09:16.935 INFO:teuthology.orchestra.run.smithi120.stdout:2878 punch from 0x1178642 to 0x117c19d, (0x3b5b bytes) 2024-09-18T01:09:16.935 INFO:teuthology.orchestra.run.smithi120.stdout:2879 writesame 0xa95342 thru 0xaa331b data_size 0x3(0xdfda bytes) 2024-09-18T01:09:16.944 INFO:teuthology.orchestra.run.smithi120.stdout:2881 read 0x306e03 thru 0x30a64b (0x3849 bytes) 2024-09-18T01:09:16.944 INFO:teuthology.orchestra.run.smithi120.stdout:2882 punch from 0xef9e24 to 0xf0035f, (0x653b bytes) 2024-09-18T01:09:16.945 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:16.945 INFO:teuthology.orchestra.run.smithi120.stdout:2886 clone 28 order 18 su 65536 sc 13 2024-09-18T01:09:16.948 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:16.946+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:16.992 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:16.992 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:17.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:17.049 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:17.047+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:17.105 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:17.105 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:17.105 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:17.114 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:17.158 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:17.157+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:17.177 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 125.319196465, 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:17.219 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:17.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65805 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 124928~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595699629999999, 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053947967999999999, 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65805 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621308+0000", 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0710000000000002e-06 2024-09-18T01:09:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621315+0000", 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9840000000000004e-06 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621320+0000", 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675017+0000", 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053696902999999997 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675064+0000", 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7305999999999997e-05 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675253+0000", 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018950400000000001 2024-09-18T01:09:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595679632, 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595658221000001, 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595635305999998, 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:17.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:17.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.244 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595614289, 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:17.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:17.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595583082000001, 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:17.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:17.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595560754000001, 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:17.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595538730000001, 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:17.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:17.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595489303000001, 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595458206, 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.268 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:17.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:17.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595435211000002, 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.277 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595413492999999, 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:17.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:17.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595392071999999, 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:17.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:17.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595370915, 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:17.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:17.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:17.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595345936000001, 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:17.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595314612999999, 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:17.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595291723999999, 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:17.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:17.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595268095000002, 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:17.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:17.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.595245913999999, 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:17.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:17.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:17.320 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:17.321 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:17.321 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:17.322 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:17.322 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:17.323 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:17.320+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:17.323 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:17.329 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:17.329 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:17.345 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:17.346 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:17.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:17.379 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:17.379 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:17.409 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:17.410 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:17.422 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:17.420+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:17.426 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:17.427 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:17.444 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:17.444 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:17.453 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:17.537 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:17.535+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:17.573 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:09:17.603 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:17.603 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:17.613 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:17.626 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:17.625+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:17.728 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:17.727+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:17.760 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:17.760 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:17.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:17.828 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:17.827+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:17.872 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:17.872 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:17.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:17.930 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:17.929+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:17.984 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:17.984 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:17.993 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:18.037 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:18.035+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:18.095 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.096 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.096 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:18.096 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:18.096 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:09:18.096 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:09:18.104 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:18.145 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:18.143+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:18.233 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:18.232+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:18.251 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.251 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.260 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:18.334 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:18.333+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:18.363 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.363 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.372 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:18.386 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:18.386 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:18.386 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 124, 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:09:17.564807+0000", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:09:00.604392+0000", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5801984, 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:09:18.387 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:09:18.388 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "jewel", 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:09:18.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:09:18.390 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:09:18.391 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:09:18.392 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:09:18.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:09:18.394 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:09:18.395 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:09:18.396 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:09:18.397 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:09:18.399 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:09:18.400 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:09:18.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:09:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:09:18.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:09:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.405 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 2, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 2, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.75, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:09:18.407 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:09:18.408 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:09:18.409 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "124", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 58, 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 124, 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:09:18.410 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:09:18.411 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:09:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.413 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.4900000095367432, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.4900000095367432, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.309999942779541, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.309999942779541, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.414 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:09:18.415 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:09:18.420 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.421 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 116, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.422 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:09:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 18, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.429 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:09:18.432 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:09:18.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 113, 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:09:18.435 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:09:18.439 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.440 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 116, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:09:18.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:09:18.445 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.446 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 113, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:09:18.448 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.449 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:09:18.451 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 82, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:09:18.454 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.457 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:09:18.458 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 51, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.464 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.466 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 65536, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.467 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:09:18.469 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:09:18.470 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:09:18.471 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 57, 2024-09-18T01:09:18.472 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:09:18.473 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:09:18.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:18.474 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:18.476 INFO:tasks.thrashosds.thrasher:j is {'epoch': 124, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:09:17.564807+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:09:00.604392+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 58, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 57, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:09:18.476 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:09:18.476 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:18.473+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:18.477 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.477 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:18.575 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:09:18.575 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:18.574+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:18.652 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.652 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:18.676 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:18.675+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:18.769 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.769 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.778 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:18.778 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:18.776+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:18.834 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:18.835 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:09:18.847 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-18T01:09:18.742858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1012123144,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":0,"num_read":118880,"num_read_kb":782424,"num_write":15878,"num_write_kb":524290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":30752621,"num_keys_recovered":22,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11131,"ondisk_log_size":11131,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":783044,"kb_used_data":493916,"kb_used_omap":10,"kb_used_meta":288949,"kb_avail":659819836,"statfs":{"total":676457349120,"available":675655512064,"internally_reserved":0,"allocated":505769984,"data_stored":497729194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":295883883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":32768},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41627556,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4439,"num_read_kb":3511,"num_write":1013,"num_write_kb":3268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000741"},"pg_stats":[{"pgid":"3.1f","version":"123'282","reported_seq":2185,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580254+0000","last_change":"2024-09-18T01:09:16.581507+0000","last_active":"2024-09-18T01:09:17.580254+0000","last_peered":"2024-09-18T01:09:17.580254+0000","last_clean":"2024-09-18T01:09:17.580254+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:09:17.580254+0000","last_undegraded":"2024-09-18T01:09:17.580254+0000","last_fullsized":"2024-09-18T01:09:17.580254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:33.297088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000462915,"stat_sum":{"num_bytes":31219468,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2109,"num_read_kb":20502,"num_write":304,"num_write_kb":20931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.1e","version":"120'191","reported_seq":1104,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.553851+0000","last_change":"2024-09-18T01:09:00.616011+0000","last_active":"2024-09-18T01:09:14.553851+0000","last_peered":"2024-09-18T01:09:14.553851+0000","last_clean":"2024-09-18T01:09:14.553851+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:09:14.553851+0000","last_undegraded":"2024-09-18T01:09:14.553851+0000","last_fullsized":"2024-09-18T01:09:14.553851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:44:12.446482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323985,"stat_sum":{"num_bytes":11629827,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":790,"num_read_kb":16934,"num_write":270,"num_write_kb":13367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"123'213","reported_seq":1624,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.583706+0000","last_change":"2024-09-18T01:09:16.569769+0000","last_active":"2024-09-18T01:09:17.583706+0000","last_peered":"2024-09-18T01:09:17.583706+0000","last_clean":"2024-09-18T01:09:17.583706+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:09:17.583706+0000","last_undegraded":"2024-09-18T01:09:17.583706+0000","last_fullsized":"2024-09-18T01:09:17.583706+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:03.309058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066575499999999997,"stat_sum":{"num_bytes":31111320,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1065,"num_read_kb":28056,"num_write":286,"num_write_kb":31180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.1c","version":"124'341","reported_seq":1658,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.624017+0000","last_change":"2024-09-18T01:09:16.569493+0000","last_active":"2024-09-18T01:09:17.624017+0000","last_peered":"2024-09-18T01:09:17.624017+0000","last_clean":"2024-09-18T01:09:17.624017+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:09:17.624017+0000","last_undegraded":"2024-09-18T01:09:17.624017+0000","last_fullsized":"2024-09-18T01:09:17.624017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:30.255027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049692699999999998,"stat_sum":{"num_bytes":26654358,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1840,"num_read_kb":25391,"num_write":477,"num_write_kb":26297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.1b","version":"123'927","reported_seq":3444,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580359+0000","last_change":"2024-09-18T01:09:16.582024+0000","last_active":"2024-09-18T01:09:17.580359+0000","last_peered":"2024-09-18T01:09:17.580359+0000","last_clean":"2024-09-18T01:09:17.580359+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:09:17.580359+0000","last_undegraded":"2024-09-18T01:09:17.580359+0000","last_fullsized":"2024-09-18T01:09:17.580359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:45.865450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092676,"stat_sum":{"num_bytes":25216087,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":6510,"num_read_kb":29066,"num_write":1372,"num_write_kb":20774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.1a","version":"110'186","reported_seq":1956,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T01:09:15.558764+0000","last_change":"2024-09-18T01:09:00.608159+0000","last_active":"2024-09-18T01:09:15.558764+0000","last_peered":"2024-09-18T01:09:15.558764+0000","last_clean":"2024-09-18T01:09:15.558764+0000","last_became_active":"2024-09-18T01:07:28.968271+0000","last_became_peered":"2024-09-18T01:07:28.968271+0000","last_unstale":"2024-09-18T01:09:15.558764+0000","last_undegraded":"2024-09-18T01:09:15.558764+0000","last_fullsized":"2024-09-18T01:09:15.558764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:53.441596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459934,"stat_sum":{"num_bytes":15999676,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1688,"num_read_kb":21637,"num_write":243,"num_write_kb":15443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"123'196","reported_seq":1892,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.579611+0000","last_change":"2024-09-18T01:09:16.570367+0000","last_active":"2024-09-18T01:09:17.579611+0000","last_peered":"2024-09-18T01:09:17.579611+0000","last_clean":"2024-09-18T01:09:17.579611+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:09:17.579611+0000","last_undegraded":"2024-09-18T01:09:17.579611+0000","last_fullsized":"2024-09-18T01:09:17.579611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:19.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046630599999999999,"stat_sum":{"num_bytes":36456314,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1362,"num_read_kb":19274,"num_write":222,"num_write_kb":14669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.18","version":"121'279","reported_seq":9072,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.573910+0000","last_change":"2024-09-18T01:09:00.615780+0000","last_active":"2024-09-18T01:09:14.573910+0000","last_peered":"2024-09-18T01:09:14.573910+0000","last_clean":"2024-09-18T01:09:14.573910+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:09:14.573910+0000","last_undegraded":"2024-09-18T01:09:14.573910+0000","last_fullsized":"2024-09-18T01:09:14.573910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:12:36.719711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912698,"stat_sum":{"num_bytes":45983297,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9153,"num_read_kb":43902,"num_write":318,"num_write_kb":17610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"123'359","reported_seq":1556,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580407+0000","last_change":"2024-09-18T01:09:16.581821+0000","last_active":"2024-09-18T01:09:17.580407+0000","last_peered":"2024-09-18T01:09:17.580407+0000","last_clean":"2024-09-18T01:09:17.580407+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:09:17.580407+0000","last_undegraded":"2024-09-18T01:09:17.580407+0000","last_fullsized":"2024-09-18T01:09:17.580407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:48.437581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069012700000000001,"stat_sum":{"num_bytes":30636262,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1638,"num_read_kb":21114,"num_write":520,"num_write_kb":15553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.16","version":"123'183","reported_seq":1337,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.583767+0000","last_change":"2024-09-18T01:09:16.569543+0000","last_active":"2024-09-18T01:09:17.583767+0000","last_peered":"2024-09-18T01:09:17.583767+0000","last_clean":"2024-09-18T01:09:17.583767+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:09:17.583767+0000","last_undegraded":"2024-09-18T01:09:17.583767+0000","last_fullsized":"2024-09-18T01:09:17.583767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:30.015949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430438,"stat_sum":{"num_bytes":26422918,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":872,"num_read_kb":14956,"num_write":216,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.15","version":"123'275","reported_seq":5813,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602946+0000","last_change":"2024-09-18T01:09:16.687268+0000","last_active":"2024-09-18T01:09:17.602946+0000","last_peered":"2024-09-18T01:09:17.602946+0000","last_clean":"2024-09-18T01:09:17.602946+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:09:17.602946+0000","last_undegraded":"2024-09-18T01:09:17.602946+0000","last_fullsized":"2024-09-18T01:09:17.602946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:10:33.843523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059000000000000003,"stat_sum":{"num_bytes":38823989,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5589,"num_read_kb":46377,"num_write":285,"num_write_kb":21077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.14","version":"122'474","reported_seq":1902,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T01:09:15.687265+0000","last_change":"2024-09-18T01:09:00.608235+0000","last_active":"2024-09-18T01:09:15.687265+0000","last_peered":"2024-09-18T01:09:15.687265+0000","last_clean":"2024-09-18T01:09:15.687265+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:09:15.687265+0000","last_undegraded":"2024-09-18T01:09:15.687265+0000","last_fullsized":"2024-09-18T01:09:15.687265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:57.544251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036472300000000001,"stat_sum":{"num_bytes":39164169,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":2707,"num_read_kb":12751,"num_write":672,"num_write_kb":8361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602420+0000","last_change":"2024-09-18T01:08:30.560455+0000","last_active":"2024-09-18T01:09:17.602420+0000","last_peered":"2024-09-18T01:09:17.602420+0000","last_clean":"2024-09-18T01:09:17.602420+0000","last_became_active":"2024-09-18T01:08:30.560107+0000","last_became_peered":"2024-09-18T01:08:30.560107+0000","last_unstale":"2024-09-18T01:09:17.602420+0000","last_undegraded":"2024-09-18T01:09:17.602420+0000","last_fullsized":"2024-09-18T01:09:17.602420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:34.598903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'274","reported_seq":1824,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580124+0000","last_change":"2024-09-18T01:09:16.581426+0000","last_active":"2024-09-18T01:09:17.580124+0000","last_peered":"2024-09-18T01:09:17.580124+0000","last_clean":"2024-09-18T01:09:17.580124+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:09:17.580124+0000","last_undegraded":"2024-09-18T01:09:17.580124+0000","last_fullsized":"2024-09-18T01:09:17.580124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:27:04.321617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000447257,"stat_sum":{"num_bytes":10354584,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2193,"num_read_kb":15265,"num_write":341,"num_write_kb":12206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":225,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580030+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:09:17.580030+0000","last_peered":"2024-09-18T01:09:17.580030+0000","last_clean":"2024-09-18T01:09:17.580030+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:09:17.580030+0000","last_undegraded":"2024-09-18T01:09:17.580030+0000","last_fullsized":"2024-09-18T01:09:17.580030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"120'271","reported_seq":3062,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.554012+0000","last_change":"2024-09-18T01:09:00.616331+0000","last_active":"2024-09-18T01:09:14.554012+0000","last_peered":"2024-09-18T01:09:14.554012+0000","last_clean":"2024-09-18T01:09:14.554012+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:09:14.554012+0000","last_undegraded":"2024-09-18T01:09:14.554012+0000","last_fullsized":"2024-09-18T01:09:14.554012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:56.560875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00129317,"stat_sum":{"num_bytes":24431257,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":3002,"num_read_kb":20351,"num_write":424,"num_write_kb":17446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602944+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:09:17.602944+0000","last_peered":"2024-09-18T01:09:17.602944+0000","last_clean":"2024-09-18T01:09:17.602944+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:09:17.602944+0000","last_undegraded":"2024-09-18T01:09:17.602944+0000","last_fullsized":"2024-09-18T01:09:17.602944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"124'206","reported_seq":2257,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.587874+0000","last_change":"2024-09-18T01:09:16.570450+0000","last_active":"2024-09-18T01:09:17.587874+0000","last_peered":"2024-09-18T01:09:17.587874+0000","last_clean":"2024-09-18T01:09:17.587874+0000","last_became_active":"2024-09-18T01:09:04.568447+0000","last_became_peered":"2024-09-18T01:09:04.568447+0000","last_unstale":"2024-09-18T01:09:17.587874+0000","last_undegraded":"2024-09-18T01:09:17.587874+0000","last_fullsized":"2024-09-18T01:09:17.587874+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:29:12.033614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042079500000000001,"stat_sum":{"num_bytes":34389897,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1850,"num_read_kb":31827,"num_write":206,"num_write_kb":13431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":10209331,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":225,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.579873+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:09:17.579873+0000","last_peered":"2024-09-18T01:09:17.579873+0000","last_clean":"2024-09-18T01:09:17.579873+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:09:17.579873+0000","last_undegraded":"2024-09-18T01:09:17.579873+0000","last_fullsized":"2024-09-18T01:09:17.579873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'353","reported_seq":8290,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.583809+0000","last_change":"2024-09-18T01:09:16.569902+0000","last_active":"2024-09-18T01:09:17.583809+0000","last_peered":"2024-09-18T01:09:17.583809+0000","last_clean":"2024-09-18T01:09:17.583809+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:09:17.583809+0000","last_undegraded":"2024-09-18T01:09:17.583809+0000","last_fullsized":"2024-09-18T01:09:17.583809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:43:42.287317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069827300000000001,"stat_sum":{"num_bytes":52813127,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":8290,"num_read_kb":50421,"num_write":578,"num_write_kb":27048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.3","version":"110'253","reported_seq":1548,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T01:09:15.558792+0000","last_change":"2024-09-18T01:09:00.608397+0000","last_active":"2024-09-18T01:09:15.558792+0000","last_peered":"2024-09-18T01:09:15.558792+0000","last_clean":"2024-09-18T01:09:15.558792+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:09:15.558792+0000","last_undegraded":"2024-09-18T01:09:15.558792+0000","last_fullsized":"2024-09-18T01:09:15.558792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:17:18.262058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043656899999999999,"stat_sum":{"num_bytes":13351605,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1688,"num_read_kb":16674,"num_write":330,"num_write_kb":10038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":226,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.583566+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:09:17.583566+0000","last_peered":"2024-09-18T01:09:17.583566+0000","last_clean":"2024-09-18T01:09:17.583566+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:09:17.583566+0000","last_undegraded":"2024-09-18T01:09:17.583566+0000","last_fullsized":"2024-09-18T01:09:17.583566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"123'203","reported_seq":8247,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.579652+0000","last_change":"2024-09-18T01:09:16.570699+0000","last_active":"2024-09-18T01:09:17.579652+0000","last_peered":"2024-09-18T01:09:17.579652+0000","last_clean":"2024-09-18T01:09:17.579652+0000","last_became_active":"2024-09-18T01:09:04.565785+0000","last_became_peered":"2024-09-18T01:09:04.565785+0000","last_unstale":"2024-09-18T01:09:17.579652+0000","last_undegraded":"2024-09-18T01:09:17.579652+0000","last_fullsized":"2024-09-18T01:09:17.579652+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:52.587963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000610635,"stat_sum":{"num_bytes":46574535,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":7771,"num_read_kb":47816,"num_write":355,"num_write_kb":20312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":12065302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.553966+0000","last_change":"2024-09-18T01:09:01.957599+0000","last_active":"2024-09-18T01:09:14.553966+0000","last_peered":"2024-09-18T01:09:14.553966+0000","last_clean":"2024-09-18T01:09:14.553966+0000","last_became_active":"2024-09-18T01:09:01.957359+0000","last_became_peered":"2024-09-18T01:09:01.957359+0000","last_unstale":"2024-09-18T01:09:14.553966+0000","last_undegraded":"2024-09-18T01:09:14.553966+0000","last_fullsized":"2024-09-18T01:09:14.553966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:26:18.897131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"123'321","reported_seq":8195,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.580080+0000","last_change":"2024-09-18T01:09:16.581812+0000","last_active":"2024-09-18T01:09:17.580080+0000","last_peered":"2024-09-18T01:09:17.580080+0000","last_clean":"2024-09-18T01:09:17.580080+0000","last_became_active":"2024-09-18T01:07:28.972003+0000","last_became_peered":"2024-09-18T01:07:28.972003+0000","last_unstale":"2024-09-18T01:09:17.580080+0000","last_undegraded":"2024-09-18T01:09:17.580080+0000","last_fullsized":"2024-09-18T01:09:17.580080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:50.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061188400000000004,"stat_sum":{"num_bytes":78038516,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":8002,"num_read_kb":24579,"num_write":343,"num_write_kb":21687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":224,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602453+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:09:17.602453+0000","last_peered":"2024-09-18T01:09:17.602453+0000","last_clean":"2024-09-18T01:09:17.602453+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:09:17.602453+0000","last_undegraded":"2024-09-18T01:09:17.602453+0000","last_fullsized":"2024-09-18T01:09:17.602453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"120'224","reported_seq":1551,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.554052+0000","last_change":"2024-09-18T01:09:00.614782+0000","last_active":"2024-09-18T01:09:14.554052+0000","last_peered":"2024-09-18T01:09:14.554052+0000","last_clean":"2024-09-18T01:09:14.554052+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:09:14.554052+0000","last_undegraded":"2024-09-18T01:09:14.554052+0000","last_fullsized":"2024-09-18T01:09:14.554052+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:38.393102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011237420000000001,"stat_sum":{"num_bytes":26950030,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1285,"num_read_kb":12825,"num_write":249,"num_write_kb":20188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":234,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.583566+0000","last_change":"2024-09-18T01:09:01.617060+0000","last_active":"2024-09-18T01:09:17.583566+0000","last_peered":"2024-09-18T01:09:17.583566+0000","last_clean":"2024-09-18T01:09:17.583566+0000","last_became_active":"2024-09-18T01:09:01.614227+0000","last_became_peered":"2024-09-18T01:09:01.614227+0000","last_unstale":"2024-09-18T01:09:17.583566+0000","last_undegraded":"2024-09-18T01:09:17.583566+0000","last_fullsized":"2024-09-18T01:09:17.583566+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:33:03.420581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":377,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602551+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:09:17.602551+0000","last_peered":"2024-09-18T01:09:17.602551+0000","last_clean":"2024-09-18T01:09:17.602551+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:09:17.602551+0000","last_undegraded":"2024-09-18T01:09:17.602551+0000","last_fullsized":"2024-09-18T01:09:17.602551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"120'568","reported_seq":1764,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.554557+0000","last_change":"2024-09-18T01:09:00.615034+0000","last_active":"2024-09-18T01:09:14.554557+0000","last_peered":"2024-09-18T01:09:14.554557+0000","last_clean":"2024-09-18T01:09:14.554557+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:09:14.554557+0000","last_undegraded":"2024-09-18T01:09:14.554557+0000","last_fullsized":"2024-09-18T01:09:14.554557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:49.392686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013303410000000001,"stat_sum":{"num_bytes":16796420,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2818,"num_read_kb":12511,"num_write":1008,"num_write_kb":9155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"120'161","reported_seq":7146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T01:09:15.687463+0000","last_change":"2024-09-18T01:09:04.557568+0000","last_active":"2024-09-18T01:09:15.687463+0000","last_peered":"2024-09-18T01:09:15.687463+0000","last_clean":"2024-09-18T01:09:15.687463+0000","last_became_active":"2024-09-18T01:09:04.556274+0000","last_became_peered":"2024-09-18T01:09:04.556274+0000","last_unstale":"2024-09-18T01:09:15.687463+0000","last_undegraded":"2024-09-18T01:09:15.687463+0000","last_fullsized":"2024-09-18T01:09:15.687463+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:59:10.842113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049415899999999998,"stat_sum":{"num_bytes":37107058,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6507,"num_read_kb":31423,"num_write":162,"num_write_kb":12153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":8477988,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"123'275","reported_seq":7838,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602512+0000","last_change":"2024-09-18T01:09:16.695103+0000","last_active":"2024-09-18T01:09:17.602512+0000","last_peered":"2024-09-18T01:09:17.602512+0000","last_clean":"2024-09-18T01:09:17.602512+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:09:17.602512+0000","last_undegraded":"2024-09-18T01:09:17.602512+0000","last_fullsized":"2024-09-18T01:09:17.602512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:50.663698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038128800000000003,"stat_sum":{"num_bytes":53756720,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":7888,"num_read_kb":28986,"num_write":312,"num_write_kb":11685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.b","version":"121'285","reported_seq":1667,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T01:09:14.597142+0000","last_change":"2024-09-18T01:09:00.614995+0000","last_active":"2024-09-18T01:09:14.597142+0000","last_peered":"2024-09-18T01:09:14.597142+0000","last_clean":"2024-09-18T01:09:14.597142+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:09:14.597142+0000","last_undegraded":"2024-09-18T01:09:14.597142+0000","last_fullsized":"2024-09-18T01:09:14.597142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:46.613752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323109,"stat_sum":{"num_bytes":16442149,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1876,"num_read_kb":20414,"num_write":384,"num_write_kb":11369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"124'620","reported_seq":2383,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602049+0000","last_change":"2024-09-18T01:09:16.569706+0000","last_active":"2024-09-18T01:09:17.602049+0000","last_peered":"2024-09-18T01:09:17.602049+0000","last_clean":"2024-09-18T01:09:17.602049+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:09:17.602049+0000","last_undegraded":"2024-09-18T01:09:17.602049+0000","last_fullsized":"2024-09-18T01:09:17.602049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:10.907207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063766300000000001,"stat_sum":{"num_bytes":13615873,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":3631,"num_read_kb":18719,"num_write":1080,"num_write_kb":12487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.d","version":"123'199","reported_seq":11332,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602839+0000","last_change":"2024-09-18T01:09:16.687037+0000","last_active":"2024-09-18T01:09:17.602839+0000","last_peered":"2024-09-18T01:09:17.602839+0000","last_clean":"2024-09-18T01:09:17.602839+0000","last_became_active":"2024-09-18T01:07:28.973854+0000","last_became_peered":"2024-09-18T01:07:28.973854+0000","last_unstale":"2024-09-18T01:09:17.602839+0000","last_undegraded":"2024-09-18T01:09:17.602839+0000","last_fullsized":"2024-09-18T01:09:17.602839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:32.556393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000357251,"stat_sum":{"num_bytes":56448980,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10975,"num_read_kb":41795,"num_write":215,"num_write_kb":18428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.e","version":"123'510","reported_seq":2718,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602644+0000","last_change":"2024-09-18T01:09:16.687505+0000","last_active":"2024-09-18T01:09:17.602644+0000","last_peered":"2024-09-18T01:09:17.602644+0000","last_clean":"2024-09-18T01:09:17.602644+0000","last_became_active":"2024-09-18T01:07:28.973815+0000","last_became_peered":"2024-09-18T01:07:28.973815+0000","last_unstale":"2024-09-18T01:09:17.602644+0000","last_undegraded":"2024-09-18T01:09:17.602644+0000","last_fullsized":"2024-09-18T01:09:17.602644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:26.629733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080504100000000005,"stat_sum":{"num_bytes":33370468,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":3341,"num_read_kb":42628,"num_write":781,"num_write_kb":29243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.f","version":"123'216","reported_seq":1541,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602772+0000","last_change":"2024-09-18T01:09:16.687514+0000","last_active":"2024-09-18T01:09:17.602772+0000","last_peered":"2024-09-18T01:09:17.602772+0000","last_clean":"2024-09-18T01:09:17.602772+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:09:17.602772+0000","last_undegraded":"2024-09-18T01:09:17.602772+0000","last_fullsized":"2024-09-18T01:09:17.602772+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:48:59.492035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058576400000000001,"stat_sum":{"num_bytes":36284229,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1248,"num_read_kb":14333,"num_write":293,"num_write_kb":10692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.10","version":"123'342","reported_seq":1284,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602474+0000","last_change":"2024-09-18T01:09:16.695149+0000","last_active":"2024-09-18T01:09:17.602474+0000","last_peered":"2024-09-18T01:09:17.602474+0000","last_clean":"2024-09-18T01:09:17.602474+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:09:17.602474+0000","last_undegraded":"2024-09-18T01:09:17.602474+0000","last_fullsized":"2024-09-18T01:09:17.602474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:37.864292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040858300000000001,"stat_sum":{"num_bytes":12451614,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1446,"num_read_kb":14666,"num_write":456,"num_write_kb":15324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.11","version":"123'225","reported_seq":1468,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602672+0000","last_change":"2024-09-18T01:09:16.686931+0000","last_active":"2024-09-18T01:09:17.602672+0000","last_peered":"2024-09-18T01:09:17.602672+0000","last_clean":"2024-09-18T01:09:17.602672+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:09:17.602672+0000","last_undegraded":"2024-09-18T01:09:17.602672+0000","last_fullsized":"2024-09-18T01:09:17.602672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:14.904503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039460999999999999,"stat_sum":{"num_bytes":53045315,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1031,"num_read_kb":10399,"num_write":328,"num_write_kb":11496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.12","version":"123'1059","reported_seq":2833,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.579908+0000","last_change":"2024-09-18T01:09:16.582118+0000","last_active":"2024-09-18T01:09:17.579908+0000","last_peered":"2024-09-18T01:09:17.579908+0000","last_clean":"2024-09-18T01:09:17.579908+0000","last_became_active":"2024-09-18T01:07:28.978142+0000","last_became_peered":"2024-09-18T01:07:28.978142+0000","last_unstale":"2024-09-18T01:09:17.579908+0000","last_undegraded":"2024-09-18T01:09:17.579908+0000","last_fullsized":"2024-09-18T01:09:17.579908+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1059,"log_dups_size":0,"ondisk_log_size":1059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:38:57.522815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089876300000000001,"stat_sum":{"num_bytes":13862708,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5945,"num_read_kb":20001,"num_write":1874,"num_write_kb":12391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"39","length":"1"}]},{"pgid":"3.13","version":"123'540","reported_seq":1586,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-18T01:09:17.602855+0000","last_change":"2024-09-18T01:09:16.687181+0000","last_active":"2024-09-18T01:09:17.602855+0000","last_peered":"2024-09-18T01:09:17.602855+0000","last_clean":"2024-09-18T01:09:17.602855+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:09:17.602855+0000","last_undegraded":"2024-09-18T01:09:17.602855+0000","last_fullsized":"2024-09-18T01:09:17.602855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:35.345372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499166,"stat_sum":{"num_bytes":22129987,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":2653,"num_read_kb":6730,"num_write":813,"num_write_kb":7994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"39","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1011532757,"num_objects":843,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":118763,"num_read_kb":782323,"num_write":15717,"num_write_kb":521872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30752621,"num_keys_recovered":22,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":525389824,"data_stored":522498357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11011,"ondisk_log_size":11011,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124452,"kb_used_data":79372,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94247388,"statfs":{"total":96636764160,"available":96509325312,"internally_reserved":0,"allocated":81276928,"data_stored":80118905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63420,"kb_used_data":32224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308420,"statfs":{"total":96636764160,"available":96571822080,"internally_reserved":0,"allocated":32997376,"data_stored":32066647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":32768},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130856,"kb_used_data":91344,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94240984,"statfs":{"total":96636764160,"available":96502767616,"internally_reserved":0,"allocated":93536256,"data_stored":92314793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509469,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81616,"kb_used_data":46260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290224,"statfs":{"total":96636764160,"available":96553189376,"internally_reserved":0,"allocated":47370240,"data_stored":46317668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118044,"kb_used_data":68804,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94253796,"statfs":{"total":96636764160,"available":96515887104,"internally_reserved":0,"allocated":70455296,"data_stored":69287929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145992,"kb_used_data":96752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94225848,"statfs":{"total":96636764160,"available":96487268352,"internally_reserved":0,"allocated":99074048,"data_stored":97778385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118664,"kb_used_data":79160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94253176,"statfs":{"total":96636764160,"available":96515252224,"internally_reserved":0,"allocated":81059840,"data_stored":79844867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":67997761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45912064,"data_stored":45617928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30916608,"data_stored":30752621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":79642624,"data_stored":79163839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97705984,"data_stored":97161821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":92057600,"data_stored":91608803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31535104,"data_stored":31366847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79220736,"data_stored":78828737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:09:18.849 INFO:tasks.thrashosds.thrasher:Setting 3.1a to [1, 6, 5, 3] 2024-09-18T01:09:18.849 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1a', '1', '6', '5', '3'] 2024-09-18T01:09:18.849 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1a 1 6 5 3 2024-09-18T01:09:18.884 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:18.884 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:18.885 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:18.882+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:18.893 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:18.979 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:18.977+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:19.009 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:19.009 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:19.018 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:19.080 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:19.078+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:19.122 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:19.122 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:19.131 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:19.216 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:19.215+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:19.276 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:19.276 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:19.282 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:19.281+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:19.284 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:19.383 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:19.382+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:19.429 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:19.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:19.437 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:19.441 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:19.485 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:19.483+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:19.561 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:19.583 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:19.583 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:19.583 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.1a pg_upmap_items mapping to [1->6,5->3] 2024-09-18T01:09:19.584 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:19.582+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:19.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:19.599 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:19.615 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:19.633 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:19.649 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:19.666 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:19.682 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:19.684 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:19.684+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:19.701 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:19.701 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:19.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:19.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67620921300001, 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:19.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:19.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:19.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.676167062, 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:19.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:19.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.676133634, 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:19.710 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:19.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67609613400001, 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:19.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67607455700001, 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:19.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.676055417, 2024-09-18T01:09:19.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:19.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67603502599999, 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:19.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67600562600001, 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:19.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.675976836, 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:19.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.675956141, 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67593665699999, 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:19.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:19.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.67590274600001, 2024-09-18T01:09:19.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.675866747, 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:19.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.59141877499999, 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:19.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:19.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:19.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.591408566, 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:19.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:19.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.091007340000004, 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:19.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:19.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.075521171000005, 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:19.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:8785 3.2 e34/23 3:45ed609b:::rbd_data.11762d3aac1.0000000000000007:head v 34'20, mlcod=32'17)", 2024-09-18T01:09:19.780 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.067630450999999, 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021624804000000001, 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631665+0000", 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.631666+0000", 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0690000000000001e-06 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640218+0000", 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.008551566 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640230+0000", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2191999999999999e-05 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640239+0000", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1169999999999993e-06 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640301+0000", 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1440999999999997e-05 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.640317+0000", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6532999999999999e-05 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653240+0000", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012923238 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.653290+0000", 2024-09-18T01:09:19.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9648000000000003e-05 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.041561447999996, 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:19.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:19.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.791 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.036392656000004, 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:19.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:19.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:19.796 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:19.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:19.798 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:19.796+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:19.898 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:19.897+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:19.942 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:19.942 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:19.942 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:19.942 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:19.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:19.998 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:19.997+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:20.054 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:20.054 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:20.054 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:20.054 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:20.054 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:20.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:20.107 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:20.106+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:20.168 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:20.168 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.595187582000001, 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:20.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:20.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:20.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:20.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.543196438999999, 2024-09-18T01:09:20.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:20.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:20.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:20.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:20.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.543119642999997, 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:20.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:20.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.538614490000001, 2024-09-18T01:09:20.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:20.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:20.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:20.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:20.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.538358694000003, 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:20.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:20.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:20.198 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.534333365999998, 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:20.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:20.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.534192216000001, 2024-09-18T01:09:20.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:20.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.533959312, 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.208 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:20.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.53134025, 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:20.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:20.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 50.509001216000001, 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:20.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:20.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558485942000001, 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.223 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:20.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:20.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558464201, 2024-09-18T01:09:20.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:20.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:20.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558443792999999, 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:20.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:20.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558424133999999, 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:20.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:20.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558390799000001, 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:20.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:20.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558356267000001, 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:20.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:20.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558311376999999, 2024-09-18T01:09:20.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:20.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:20.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558285002000002, 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:20.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:20.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558265851000002, 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:20.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.558247283, 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:20.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.267 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:20.269 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:20.269 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:20.268+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:20.370 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:20.369+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:20.412 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:20.412 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:20.412 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:20.412 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:20.421 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:20.471 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:20.470+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:20.525 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:20.525 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:20.525 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:20.525 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:20.525 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:20.534 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:20.578 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:20.577+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:20.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:20.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.74047237100001, 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.641 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:20.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:20.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65805 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 124928~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016975536, 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053947967999999999, 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65805 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621306+0000", 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:20.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621308+0000", 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0710000000000002e-06 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621315+0000", 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9840000000000004e-06 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621320+0000", 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675017+0000", 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053696902999999997 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675064+0000", 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7305999999999997e-05 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675253+0000", 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018950400000000001 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016955538000001, 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:20.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:20.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016934126999999, 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:20.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016911212, 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:20.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016890194999998, 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:20.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:20.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016858987999999, 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:20.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016836659999999, 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:20.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:20.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016814635999999, 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016765208999999, 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016734112000002, 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016711117, 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016689399000001, 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016667978000001, 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016646820999998, 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016621841999999, 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016590519000001, 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016567630000001, 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016544001, 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.016521820000001, 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:20.737 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:20.738 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:20.737+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:20.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:20.839 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:20.838+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:20.843 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:20.843 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:20.844 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:20.844 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:20.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:20.940 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:20.939+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:20.955 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:20.956 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:20.964 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:21.040 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:21.039+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.46966792800001, 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:21.068 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:21.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:21.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:21.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:21.074 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.408065306, 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:21.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:21.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:21.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:21.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.343215917999999, 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:21.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101741279999999, 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:21.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:21.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.092 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101717115, 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:21.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:21.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101696007000001, 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.097 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:21.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101675134000001, 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:21.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.106 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101656907999999, 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:21.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101564738, 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:21.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.115 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101536327000002, 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101514336000001, 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101494355, 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:21.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:21.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.101473498000001, 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:21.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096697857999999, 2024-09-18T01:09:21.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:21.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:21.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096634042000002, 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:21.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096608797999998, 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:21.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.148 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096570051, 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:21.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:21.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096547736000002, 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:21.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:21.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096525226000001, 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.158 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:21.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:21.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.096488916999999, 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.167 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:21.168 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:21.169 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:21.167+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:21.269 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:21.268+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:21.312 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:21.312 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:21.312 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:21.312 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:21.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:21.369 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:21.368+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:21.424 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:21.425 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:21.425 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:21.425 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:21.425 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:21.433 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:21.470 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:21.469+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:21.537 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 106.357126124, 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 106.34834417499999, 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758819245000002, 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758796943, 2024-09-18T01:09:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758775989, 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758755145999999, 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758733744000001, 2024-09-18T01:09:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758714564000002, 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758695208999999, 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758656604999999, 2024-09-18T01:09:21.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:21.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758626465999999, 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758603012999998, 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758580996999999, 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758560029000002, 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758539170999999, 2024-09-18T01:09:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758510722, 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758488581000002, 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758465225999998, 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.758442292000002, 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.755403294000001, 2024-09-18T01:09:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:21.636 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:21.638 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:21.638 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:09:21.639 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:21.636+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:21.694 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:21.738 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:21.737+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:21.784 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:21.785 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:21.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:21.839 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:21.838+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:21.846 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:21.846 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:21.863 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:21.864 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:21.880 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:21.897 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:21.898 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:21.904 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:21.904 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:21.907 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-18T01:09:21.919 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:21.919 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:21.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:21.939 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:21.939 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:21.940 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:21.938+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:21.958 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:21.958 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:21.976 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:21.977 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:22.037 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.038 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.040 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:22.039+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:22.047 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:22.104 INFO:teuthology.orchestra.run.smithi120.stdout:2887 write 0xc0c2659 thru 0xc0c51f7 (0x2b9f bytes) 2024-09-18T01:09:22.108 INFO:teuthology.orchestra.run.smithi120.stdout:2888 clone 29 order 22 su 524288 sc 12 2024-09-18T01:09:22.141 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:22.140+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:22.192 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.192 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:22.243 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:22.242+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:22.344 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.344 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.344 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:22.343+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:22.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:22.445 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:22.443+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:22.455 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.455 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:22.545 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:22.544+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:22.574 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.575 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:22.646 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:22.645+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:22.689 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.689 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:22.747 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:22.746+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:22.802 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.803 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:22.848 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:22.847+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:22.949 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:22.948+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:22.956 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:22.956 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:22.965 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:23.050 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:23.049+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:23.068 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.068 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:23.151 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:23.150+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:23.181 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.181 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:23.252 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:23.250+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:23.293 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.293 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:23.352 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:23.351+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:23.405 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.406 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.414 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:23.453 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:23.452+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:23.554 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:23.553+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:23.558 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.558 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.568 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:23.639 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:09:23.655 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:23.654+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:23.713 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.714 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:23.756 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:23.755+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:23.857 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:23.856+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:23.865 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:23.865 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:23.875 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:23.958 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:23.957+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:23.992 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:24.019 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:24.019 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:24.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:24.080 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:24.079+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:24.114 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:24.132 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:24.144 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:24.144 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:24.144 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:24.144 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:24.148 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:24.148 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:24.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:24.159 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:24.158+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:24.165 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:24.182 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:24.200 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:24.216 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:24.233 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:24.234 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:24.269 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:24.270 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:24.270 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:24.270 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:24.270 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:24.270 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:24.269+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:24.278 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:24.362 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:24.361+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:24.422 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.824557316, 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:24.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:24.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.762954694, 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:24.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:24.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:24.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.698105306000002, 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:24.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:24.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:24.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.443 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456630667999999, 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.444 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:24.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.448 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456606503, 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.449 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:24.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456585395000001, 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:24.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456564522000001, 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:24.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:24.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456546295999999, 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:24.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:24.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:24.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456454126000001, 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:24.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:24.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456425715000002, 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456403724000001, 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456383743, 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456362886000001, 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451587245999999, 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451523430000002, 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451498185999998, 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451459439000001, 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451437124000002, 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451414614000001, 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.451378304999999, 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:24.536 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:24.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:24.538 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:24.536+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:24.559 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:09:24.601 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:24.601 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:24.601 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:09:24.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-18T01:09:24.660 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:24.659+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:24.683 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:24.683 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:24.692 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:24.739 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:24.738+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:24.839 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:24.838+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:24.851 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:24.851 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:24.851 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:24.851 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:24.851 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:24.860 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:24.941 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:24.940+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:24.995 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:avg 11.7143 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 2.81396 -> 2.81396 (expected baseline 3.16872) 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.6 with 7 -> 7 pgs (0.597561 -> 0.597561 * mean) 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 16 -> 16 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:oload 1000 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 1 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0012 2024-09-18T01:09:24.996 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0119 2024-09-18T01:09:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:09:25.005 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.82542954199999, 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.816647593, 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.012 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:25.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227122662999999, 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227100361000002, 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:25.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227079407000002, 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:25.027 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:25.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227058564, 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:25.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:25.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:25.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227037161999998, 2024-09-18T01:09:25.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:25.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:25.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.227017982, 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:25.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226998627, 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:25.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:25.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226960023, 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.050 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:25.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:25.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:25.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226929884, 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:25.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.060 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226906431, 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:25.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:25.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:25.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226884415000001, 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:25.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:25.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226863446999999, 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:25.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226842589, 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:25.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:25.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226814139999998, 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.080 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226791999, 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:25.084 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:25.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:25.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226768644, 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.089 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:25.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:25.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.226745709999999, 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.223706711999998, 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:25.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:25.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.102 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.103 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:09:25.104 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-18T01:09:25.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:25.105 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:25.102+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:25.204 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:25.203+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:25.253 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.253 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:25.253 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:25.253 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:25.305 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:25.304+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:25.406 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:25.405+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:25.407 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.407 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:25.407 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:25.407 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:25.407 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:25.433 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:09:25.433 INFO:teuthology.orchestra.run.smithi120.stdout:avg 11.7143 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 2.81396 -> 2.81396 (expected baseline 3.16872) 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.6 with 7 -> 7 pgs (0.597561 -> 0.597561 * mean) 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 16 -> 16 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:oload 1000 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 0.05 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0012 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0119 2024-09-18T01:09:25.434 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:09:25.447 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:09:25.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-18T01:09:25.507 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:25.506+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 126.600300256, 2024-09-18T01:09:25.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:25.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:09:25.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.33210493999999, 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:09:25.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:09:25.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.33050919199999, 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:09:25.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:09:25.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.329698573, 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:09:25.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.36496461900001, 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:09:25.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.36230791600001, 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:09:25.565 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.779660372999999, 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:09:25.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.365346914, 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:09:25.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:09:25.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.863647892000003, 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.862071005000004, 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:09:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.832323415000005, 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:09:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:09:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:09:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.676943929000004, 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:09:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:09:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.673290499000004, 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:25.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:09:25.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.957623824000002, 2024-09-18T01:09:25.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:09:25.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 866 entries 100'101 (0'0) error 3:9153dd03:::rbd_data.117649d3c4d0.0000000000000093:head by client.4470.0:61161 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'99)", 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.111400650999997, 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048029769999999999, 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428633+0000", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7910000000000005e-06 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428653+0000", 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109999999999999e-05 2024-09-18T01:09:25.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428667+0000", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4187e-05 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428764+0000", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.433427+0000", 2024-09-18T01:09:25.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046639020000000002 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 869 entries 100'103 (0'0) error 3:94e57bb7:::rbd_data.117649d3c4d0.00000000000000b2:head by client.4470.0:61192 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.098113873000003, 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047167169999999996, 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:25.611 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441912+0000", 2024-09-18T01:09:25.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441920+0000", 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2260000000000005e-06 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441942+0000", 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1804e-05 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441957+0000", 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4599e-05 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.442024+0000", 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7860000000000004e-05 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.446628+0000", 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046035479999999998 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.614 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 871 entries 100'104 (0'0) error 3:90b555ad:::rbd_data.117649d3c4d0.00000000000000ba:head by client.4470.0:61200 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.095586953999998, 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049806420000000004, 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444439+0000", 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:09:25.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444445+0000", 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444459+0000", 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4241e-05 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444468+0000", 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5399999999999996e-06 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444541+0000", 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.449419+0000", 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048779060000000004 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116214 3.4 e110/23 3:2420fb28:::rbd_data.11766701b08b.0000000000000277:head v 110'180, mlcod=108'178)", 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.484937600999999, 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045364760000000002, 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999997e-07 2024-09-18T01:09:25.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055097+0000", 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.208e-06 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055111+0000", 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545999999999999e-05 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055118+0000", 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.934e-06 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055175+0000", 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7240000000000001e-05 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055201+0000", 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5978000000000001e-05 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059539+0000", 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043376889999999996 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059624+0000", 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5118999999999996e-05 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.623 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116558 3.9 e110/23 3:9119117f:::rbd_data.11766701b08b.0000000000000127:head v 110'143, mlcod=110'141)", 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.332119625000001, 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005246901, 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207914+0000", 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9929999999999995e-06 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207925+0000", 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0828e-05 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207934+0000", 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0860000000000008e-06 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207976+0000", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2274000000000002e-05 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207991+0000", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4296e-05 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213113+0000", 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051225649999999999 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213152+0000", 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9116999999999998e-05 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116699 3.4 e110/23 3:27f7a2f6:::rbd_data.11766701b08b.00000000000001bf:head v 110'196, mlcod=110'193)", 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.27349723, 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042393159999999999, 2024-09-18T01:09:25.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:25.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266529+0000", 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266534+0000", 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5500000000000002e-06 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266545+0000", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0926e-05 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266551+0000", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3049999999999998e-06 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266596+0000", 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4864999999999998e-05 2024-09-18T01:09:25.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266617+0000", 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0682e-05 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270721+0000", 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041039140000000002 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:25.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270767+0000", 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.634e-05 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:25.633 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.635 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:25.635 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:25.633+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:25.735 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:25.735+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:25.746 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.746 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:25.746 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:25.746 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.755 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:25.756 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-18T01:09:25.794 INFO:teuthology.orchestra.run.smithi120.stdout:2889 punch from 0xbc7d4be to 0xbc86ea5, (0x99e7 bytes) 2024-09-18T01:09:25.798 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.798 INFO:teuthology.orchestra.run.smithi120.stdout:2893 writesame 0x38ae484 thru 0x38b6032 data_size 0x1(0x7baf bytes) 2024-09-18T01:09:25.813 INFO:teuthology.orchestra.run.smithi120.stdout:2894 read 0x7c29d38 thru 0x7c29f8a (0x253 bytes) 2024-09-18T01:09:25.814 INFO:teuthology.orchestra.run.smithi120.stdout:2895 read 0x1788ea8 thru 0x178b917 (0x2a70 bytes) 2024-09-18T01:09:25.824 INFO:teuthology.orchestra.run.smithi120.stdout:2898 write 0xe8b4d09 thru 0xe8c22a6 (0xd59e bytes) 2024-09-18T01:09:25.828 INFO:teuthology.orchestra.run.smithi120.stdout:2899 punch from 0xb657b27 to 0xb664eb6, (0xd38f bytes) 2024-09-18T01:09:25.828 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.828 INFO:teuthology.orchestra.run.smithi120.stdout:2902 read 0x570de09 thru 0x571cfb4 (0xf1ac bytes) 2024-09-18T01:09:25.828 INFO:teuthology.orchestra.run.smithi120.stdout:2903 punch from 0x76bd718 to 0x76c6fed, (0x98d5 bytes) 2024-09-18T01:09:25.829 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.829 INFO:teuthology.orchestra.run.smithi120.stdout:2906 read 0x33aaf3 thru 0x34a12c (0xf63a bytes) 2024-09-18T01:09:25.829 INFO:teuthology.orchestra.run.smithi120.stdout:2907 trunc from 0xe8c22a7 to 0xe891ee9 2024-09-18T01:09:25.836 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:25.835+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:25.844 INFO:teuthology.orchestra.run.smithi120.stdout:2908 trunc from 0xe891ee9 to 0x6f9387c 2024-09-18T01:09:25.855 INFO:teuthology.orchestra.run.smithi120.stdout:2909 read 0x2518ca8 thru 0x2523798 (0xaaf1 bytes) 2024-09-18T01:09:25.855 INFO:teuthology.orchestra.run.smithi120.stdout:2911 writesame 0x42a0f21 thru 0x42a3711 data_size 0x5(0x27f1 bytes) 2024-09-18T01:09:25.866 INFO:teuthology.orchestra.run.smithi120.stdout:2913 write 0x837d629 thru 0x838ce54 (0xf82c bytes) 2024-09-18T01:09:25.877 INFO:teuthology.orchestra.run.smithi120.stdout:2914 write 0x88aa831 thru 0x88afd88 (0x5558 bytes) 2024-09-18T01:09:25.880 INFO:teuthology.orchestra.run.smithi120.stdout:2916 read 0x4b77b7f thru 0x4b81df2 (0xa274 bytes) 2024-09-18T01:09:25.881 INFO:teuthology.orchestra.run.smithi120.stdout:2917 writesame 0x4971d5a thru 0x4973ee3 data_size 0x9(0x218a bytes) 2024-09-18T01:09:25.884 INFO:teuthology.orchestra.run.smithi120.stdout:2918 read 0x80bad3e thru 0x80c2df6 (0x80b9 bytes) 2024-09-18T01:09:25.885 INFO:teuthology.orchestra.run.smithi120.stdout:2920 read 0x12cda5b thru 0x12d6603 (0x8ba9 bytes) 2024-09-18T01:09:25.885 INFO:teuthology.orchestra.run.smithi120.stdout:2921 read 0x27d32b0 thru 0x27dc2b9 (0x900a bytes) 2024-09-18T01:09:25.885 INFO:teuthology.orchestra.run.smithi120.stdout:2923 punch from 0x30aa8e0 to 0x30b35d8, (0x8cf8 bytes) 2024-09-18T01:09:25.887 INFO:teuthology.orchestra.run.smithi120.stdout:2925 trunc from 0x88afd89 to 0xa09082d 2024-09-18T01:09:25.903 INFO:teuthology.orchestra.run.smithi120.stdout:2926 write 0xdc52c79 thru 0xdc5d036 (0xa3be bytes) 2024-09-18T01:09:25.907 INFO:teuthology.orchestra.run.smithi120.stdout:2927 write 0xacebd9c thru 0xacf1be9 (0x5e4e bytes) 2024-09-18T01:09:25.909 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:25.909 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:25.909 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:25.909 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:25.909 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:25.912 INFO:teuthology.orchestra.run.smithi120.stdout:2928 punch from 0x1d2c992 to 0x1d30c19, (0x4287 bytes) 2024-09-18T01:09:25.912 INFO:teuthology.orchestra.run.smithi120.stdout:2930 punch from 0x11a95c to 0x121d3f, (0x73e3 bytes) 2024-09-18T01:09:25.912 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.913 INFO:teuthology.orchestra.run.smithi120.stdout:2933 trunc from 0xdc5d037 to 0xcd6e96c 2024-09-18T01:09:25.920 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:25.920 INFO:teuthology.orchestra.run.smithi120.stdout:2934 write 0xdcb2fb thru 0xdd928d (0xdf93 bytes) 2024-09-18T01:09:25.923 INFO:teuthology.orchestra.run.smithi120.stdout:2935 read 0x441ac5b thru 0x442796c (0xcd12 bytes) 2024-09-18T01:09:25.924 INFO:teuthology.orchestra.run.smithi120.stdout:2936 trunc from 0xcd6e96c to 0xc064445 2024-09-18T01:09:25.935 INFO:teuthology.orchestra.run.smithi120.stdout:2937 punch from 0x4d0d244 to 0x4d14fff, (0x7dbb bytes) 2024-09-18T01:09:25.935 INFO:teuthology.orchestra.run.smithi120.stdout:2938 punch from 0x676901c to 0x67718e9, (0x88cd bytes) 2024-09-18T01:09:25.935 INFO:teuthology.orchestra.run.smithi120.stdout:2939 write 0xae9cd69 thru 0xaeaaf3a (0xe1d2 bytes) 2024-09-18T01:09:25.936 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:25.935+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:25.939 INFO:teuthology.orchestra.run.smithi120.stdout:2942 read 0x1a5aaab thru 0x1a5b149 (0x69f bytes) 2024-09-18T01:09:25.939 INFO:teuthology.orchestra.run.smithi120.stdout:2943 write 0x10a339a thru 0x10b22af (0xef16 bytes) 2024-09-18T01:09:25.941 INFO:teuthology.orchestra.run.smithi120.stdout:2944 write 0x5227d65 thru 0x5236f20 (0xf1bc bytes) 2024-09-18T01:09:25.945 INFO:teuthology.orchestra.run.smithi120.stdout:2945 read 0xa7895c8 thru 0xa78b588 (0x1fc1 bytes) 2024-09-18T01:09:25.945 INFO:teuthology.orchestra.run.smithi120.stdout:2947 read 0x3cefc9f thru 0x3cf9f4c (0xa2ae bytes) 2024-09-18T01:09:25.945 INFO:teuthology.orchestra.run.smithi120.stdout:2948 read 0x28e0614 thru 0x28e5218 (0x4c05 bytes) 2024-09-18T01:09:25.946 INFO:teuthology.orchestra.run.smithi120.stdout:2950 read 0x9f4a9c5 thru 0x9f4e233 (0x386f bytes) 2024-09-18T01:09:25.946 INFO:teuthology.orchestra.run.smithi120.stdout:2951 write 0xb05179f thru 0xb05d2fd (0xbb5f bytes) 2024-09-18T01:09:25.950 INFO:teuthology.orchestra.run.smithi120.stdout:2952 write 0x5dfb204 thru 0x5e038e4 (0x86e1 bytes) 2024-09-18T01:09:25.953 INFO:teuthology.orchestra.run.smithi120.stdout:2958 read 0x24987d0 thru 0x249dfe7 (0x5818 bytes) 2024-09-18T01:09:25.954 INFO:teuthology.orchestra.run.smithi120.stdout:2959 write 0x17772d7 thru 0x177f625 (0x834f bytes) 2024-09-18T01:09:25.965 INFO:teuthology.orchestra.run.smithi120.stdout:2960 writesame 0xc957594 thru 0xc95f7e3 data_size 0x78(0x8250 bytes) 2024-09-18T01:09:25.966 INFO:teuthology.orchestra.run.smithi120.stdout:2961 write 0x9c25bc1 thru 0x9c3040d (0xa84d bytes) 2024-09-18T01:09:25.969 INFO:teuthology.orchestra.run.smithi120.stdout:2963 writesame 0xb613dc thru 0xb68189 data_size 0x2(0x6dae bytes) 2024-09-18T01:09:25.976 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.976 INFO:teuthology.orchestra.run.smithi120.stdout:2965 write 0xbb7f1ec thru 0xbb7fed9 (0xcee bytes) 2024-09-18T01:09:25.980 INFO:teuthology.orchestra.run.smithi120.stdout:2967 read 0x20508ca thru 0x20552a4 (0x49db bytes) 2024-09-18T01:09:25.980 INFO:teuthology.orchestra.run.smithi120.stdout:2968 read 0x1d3b593 thru 0x1d43822 (0x8290 bytes) 2024-09-18T01:09:25.984 INFO:teuthology.orchestra.run.smithi120.stdout:2969 trunc from 0xc95f7e4 to 0x9676080 2024-09-18T01:09:25.996 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:25.996 INFO:teuthology.orchestra.run.smithi120.stdout:2971 writesame 0x1cd78c3 thru 0x1cda938 data_size 0x1(0x3076 bytes) 2024-09-18T01:09:26.011 INFO:teuthology.orchestra.run.smithi120.stdout:2973 writesame 0xbff969c thru 0xc0032cc data_size 0xb(0x9c31 bytes) 2024-09-18T01:09:26.011 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.011 INFO:teuthology.orchestra.run.smithi120.stdout:2975 write 0xb31f9d4 thru 0xb32f03c (0xf669 bytes) 2024-09-18T01:09:26.015 INFO:teuthology.orchestra.run.smithi120.stdout:2976 trunc from 0xc0032cd to 0xd05a857 2024-09-18T01:09:26.020 INFO:teuthology.orchestra.run.smithi120.stdout:2977 write 0x1c5c5dc thru 0x1c5fb11 (0x3536 bytes) 2024-09-18T01:09:26.024 INFO:teuthology.orchestra.run.smithi120.stdout:2980 punch from 0xc9fca41 to 0xca058f1, (0x8eb0 bytes) 2024-09-18T01:09:26.024 INFO:teuthology.orchestra.run.smithi120.stdout:2981 read 0x996dec2 thru 0x9972816 (0x4955 bytes) 2024-09-18T01:09:26.024 INFO:teuthology.orchestra.run.smithi120.stdout:2982 write 0x44f8209 thru 0x44fa3b3 (0x21ab bytes) 2024-09-18T01:09:26.025 INFO:teuthology.orchestra.run.smithi120.stdout:2984 write 0x73864d5 thru 0x7393b19 (0xd645 bytes) 2024-09-18T01:09:26.030 INFO:teuthology.orchestra.run.smithi120.stdout:2989 write 0x295153c thru 0x295a92f (0x93f4 bytes) 2024-09-18T01:09:26.034 INFO:teuthology.orchestra.run.smithi120.stdout:2990 read 0x5de0f3a thru 0x5de848e (0x7555 bytes) 2024-09-18T01:09:26.034 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.034 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.034 INFO:teuthology.orchestra.run.smithi120.stdout:2996 read 0x4f5a684 thru 0x4f5bd15 (0x1692 bytes) 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405795429000001, 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:26.035 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:26.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:26.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:26.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405770443999998, 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:26.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:26.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405750664999999, 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:26.044 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:26.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:26.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405724263000003, 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:26.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:26.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:26.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405702767000001, 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:26.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.057 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405683471000003, 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.058 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:26.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405665636999998, 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.063 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.063 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:26.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405639573999999, 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.068 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405620218999999, 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.073 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:26.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405589974999998, 2024-09-18T01:09:26.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:26.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:26.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405571379999998, 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:26.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.086 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405552821000001, 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:26.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:26.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405534766999999, 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.091 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:26.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:26.095 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405517007999997, 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:26.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:26.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405490739000001, 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.100 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:26.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:26.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405472119000002, 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.105 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:26.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.405453868999999, 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.113 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.404902493999998, 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:26.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:26.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:26.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.403303508, 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:26.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout:2998 write 0x350daa thru 0x35899c (0x7bf3 bytes) 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:26.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.122 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.399537520999999, 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:26.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:26.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:26.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:26.127 INFO:teuthology.orchestra.run.smithi120.stdout:2999 trunc from 0xd05a857 to 0x5a56548 2024-09-18T01:09:26.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:26.132 INFO:teuthology.orchestra.run.smithi120.stdout:3002 write 0x4a28c6f thru 0x4a2f463 (0x67f5 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3004 read 0x3e20118 thru 0x3e28e2c (0x8d15 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3005 write 0x68377ca thru 0x6846cc1 (0xf4f8 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3009 writesame 0x53add06 thru 0x53b7b73 data_size 0x7(0x9e6e bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3011 read 0x67f23f1 thru 0x67fbbc6 (0x97d6 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3016 punch from 0x162e62a to 0x16377db, (0x91b1 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3018 writesame 0xc50f81e thru 0xc51ac33 data_size 0xe(0xb416 bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3020 trunc from 0xc51ac34 to 0x906d154 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3022 read 0x1f661d6 thru 0x1f7225f (0xc08a bytes) 2024-09-18T01:09:26.133 INFO:teuthology.orchestra.run.smithi120.stdout:3023 write 0x32be6aa thru 0x32c1d9d (0x36f4 bytes) 2024-09-18T01:09:26.134 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.134 INFO:teuthology.orchestra.run.smithi120.stdout:3025 writesame 0x242a698 thru 0x242f900 data_size 0x11(0x5269 bytes) 2024-09-18T01:09:26.134 INFO:teuthology.orchestra.run.smithi120.stdout:3026 trunc from 0x906d154 to 0xd1a8a87 2024-09-18T01:09:26.134 INFO:teuthology.orchestra.run.smithi120.stdout:3027 write 0x9b56d0b thru 0x9b58bee (0x1ee4 bytes) 2024-09-18T01:09:26.134 INFO:teuthology.orchestra.run.smithi120.stdout:3028 write 0x10e5b6d thru 0x10ea48c (0x4920 bytes) 2024-09-18T01:09:26.134 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:26.129+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:26.135 INFO:teuthology.orchestra.run.smithi120.stdout:3029 punch from 0x22f6a03 to 0x22f93a8, (0x29a5 bytes) 2024-09-18T01:09:26.135 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.135 INFO:teuthology.orchestra.run.smithi120.stdout:3033 write 0x2746dc6 thru 0x2753247 (0xc482 bytes) 2024-09-18T01:09:26.135 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.135 INFO:teuthology.orchestra.run.smithi120.stdout:3035 write 0x88534e2 thru 0x88561a5 (0x2cc4 bytes) 2024-09-18T01:09:26.136 INFO:teuthology.orchestra.run.smithi120.stdout:3036 read 0xd04fd7f thru 0xd0532bf (0x3541 bytes) 2024-09-18T01:09:26.136 INFO:teuthology.orchestra.run.smithi120.stdout:3037 punch from 0x107c0a8 to 0x107d659, (0x15b1 bytes) 2024-09-18T01:09:26.137 INFO:teuthology.orchestra.run.smithi120.stdout:3038 read 0xa94750c thru 0xa94ac51 (0x3746 bytes) 2024-09-18T01:09:26.137 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.137 INFO:teuthology.orchestra.run.smithi120.stdout:3041 write 0x7d78fa1 thru 0x7d84620 (0xb680 bytes) 2024-09-18T01:09:26.141 INFO:teuthology.orchestra.run.smithi120.stdout:3042 read 0x6453dfc thru 0x645c7d5 (0x89da bytes) 2024-09-18T01:09:26.142 INFO:teuthology.orchestra.run.smithi120.stdout:3043 read 0xbf6ee3a thru 0xbf74a78 (0x5c3f bytes) 2024-09-18T01:09:26.142 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.142 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.142 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.142 INFO:teuthology.orchestra.run.smithi120.stdout:3048 read 0x17e349b thru 0x17f1e37 (0xe99d bytes) 2024-09-18T01:09:26.148 INFO:teuthology.orchestra.run.smithi120.stdout:3051 write 0xd293e27 thru 0xd29843a (0x4614 bytes) 2024-09-18T01:09:26.152 INFO:teuthology.orchestra.run.smithi120.stdout:3052 read 0x1c4c8c thru 0x1cb169 (0x64de bytes) 2024-09-18T01:09:26.152 INFO:teuthology.orchestra.run.smithi120.stdout:3053 writesame 0x580d24c thru 0x581a4bd data_size 0x7b(0xd272 bytes) 2024-09-18T01:09:26.154 INFO:teuthology.orchestra.run.smithi120.stdout:3054 punch from 0x867cfde to 0x8682540, (0x5562 bytes) 2024-09-18T01:09:26.154 INFO:teuthology.orchestra.run.smithi120.stdout:3055 read 0x87a6b80 thru 0x87b0e67 (0xa2e8 bytes) 2024-09-18T01:09:26.154 INFO:teuthology.orchestra.run.smithi120.stdout:3056 write 0x22da483 thru 0x22e3cdf (0x985d bytes) 2024-09-18T01:09:26.155 INFO:teuthology.orchestra.run.smithi120.stdout:3057 read 0x115c7d7 thru 0x1162bb6 (0x63e0 bytes) 2024-09-18T01:09:26.156 INFO:teuthology.orchestra.run.smithi120.stdout:3058 read 0x8a8651f thru 0x8a92ab5 (0xc597 bytes) 2024-09-18T01:09:26.162 INFO:teuthology.orchestra.run.smithi120.stdout:3059 trunc from 0xd29843b to 0x7d65755 2024-09-18T01:09:26.174 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.175 INFO:teuthology.orchestra.run.smithi120.stdout:3061 trunc from 0x7d65755 to 0x7bd35d9 2024-09-18T01:09:26.185 INFO:teuthology.orchestra.run.smithi120.stdout:3062 trunc from 0x7bd35d9 to 0xbc49040 2024-09-18T01:09:26.200 INFO:teuthology.orchestra.run.smithi120.stdout:3063 trunc from 0xbc49040 to 0x52e50e4 2024-09-18T01:09:26.230 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:26.229+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:26.231 INFO:teuthology.orchestra.run.smithi120.stdout:3064 write 0x8f9d0e5 thru 0x8fabe51 (0xed6d bytes) 2024-09-18T01:09:26.234 INFO:teuthology.orchestra.run.smithi120.stdout:3065 read 0x2a91498 thru 0x2a924d4 (0x103d bytes) 2024-09-18T01:09:26.234 INFO:teuthology.orchestra.run.smithi120.stdout:3067 punch from 0x2026a7c to 0x202dd06, (0x728a bytes) 2024-09-18T01:09:26.234 INFO:teuthology.orchestra.run.smithi120.stdout:3068 write 0x437f49c thru 0x438bb32 (0xc697 bytes) 2024-09-18T01:09:26.238 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.238 INFO:teuthology.orchestra.run.smithi120.stdout:3070 read 0x439ba4c thru 0x439f15d (0x3712 bytes) 2024-09-18T01:09:26.239 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.239 INFO:teuthology.orchestra.run.smithi120.stdout:3072 write 0x317ec6d thru 0x318c7c0 (0xdb54 bytes) 2024-09-18T01:09:26.241 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:26.241 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:26.241 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:26.242 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:26.242 INFO:teuthology.orchestra.run.smithi120.stdout:3074 writesame 0x853e832 thru 0x8549a0e data_size 0x1(0xb1dd bytes) 2024-09-18T01:09:26.245 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:26.251 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:26.265 INFO:teuthology.orchestra.run.smithi120.stdout:3075 writesame 0x18f8324 thru 0x19012ce data_size 0x1(0x8fab bytes) 2024-09-18T01:09:26.288 INFO:teuthology.orchestra.run.smithi120.stdout:3076 trunc from 0x8fabe52 to 0x4d76a98 2024-09-18T01:09:26.302 INFO:teuthology.orchestra.run.smithi120.stdout:3078 write 0x323ee8 thru 0x333027 (0xf140 bytes) 2024-09-18T01:09:26.309 INFO:teuthology.orchestra.run.smithi120.stdout:3082 trunc from 0x4d76a98 to 0x9e436ff 2024-09-18T01:09:26.316 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.325 INFO:teuthology.orchestra.run.smithi120.stdout:3084 trunc from 0x9e436ff to 0x25f63b7 2024-09-18T01:09:26.331 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:26.330+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:26.349 INFO:teuthology.orchestra.run.smithi120.stdout:3085 punch from 0x41a3a3 to 0x4225ba, (0x8217 bytes) 2024-09-18T01:09:26.349 INFO:teuthology.orchestra.run.smithi120.stdout:3086 read 0x80f794 thru 0x812b40 (0x33ad bytes) 2024-09-18T01:09:26.361 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:26.361 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:26.361 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:26.361 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:26.362 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:26.367 INFO:teuthology.orchestra.run.smithi120.stdout:3087 write 0xdc91d9b thru 0xdc9b982 (0x9be8 bytes) 2024-09-18T01:09:26.372 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:26.373 INFO:teuthology.orchestra.run.smithi120.stdout:3089 writesame 0x790b76d thru 0x790f6b5 data_size 0x1(0x3f49 bytes) 2024-09-18T01:09:26.379 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:26.379 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:26.382 INFO:teuthology.orchestra.run.smithi120.stdout:3090 read 0xd18bf61 thru 0xd199178 (0xd218 bytes) 2024-09-18T01:09:26.382 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.382 INFO:teuthology.orchestra.run.smithi120.stdout:3092 write 0xa0595a8 thru 0xa05d12e (0x3b87 bytes) 2024-09-18T01:09:26.386 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.386 INFO:teuthology.orchestra.run.smithi120.stdout:3094 read 0x5432e6b thru 0x5434c2b (0x1dc1 bytes) 2024-09-18T01:09:26.386 INFO:teuthology.orchestra.run.smithi120.stdout:3097 punch from 0xa96d51 to 0xa98584, (0x1833 bytes) 2024-09-18T01:09:26.386 INFO:teuthology.orchestra.run.smithi120.stdout:3100 write 0x740f81c thru 0x741f728 (0xff0d bytes) 2024-09-18T01:09:26.389 INFO:teuthology.orchestra.run.smithi120.stdout:3101 read 0x6cee408 thru 0x6cf2f34 (0x4b2d bytes) 2024-09-18T01:09:26.389 INFO:teuthology.orchestra.run.smithi120.stdout:3103 writesame 0xd3adbf5 thru 0xd3b5982 data_size 0xb(0x7d8e bytes) 2024-09-18T01:09:26.393 INFO:teuthology.orchestra.run.smithi120.stdout:3104 read 0xa5df907 thru 0xa5e3c37 (0x4331 bytes) 2024-09-18T01:09:26.393 INFO:teuthology.orchestra.run.smithi120.stdout:3105 write 0xc179135 thru 0xc17e814 (0x56e0 bytes) 2024-09-18T01:09:26.398 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:26.398 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:26.399 INFO:teuthology.orchestra.run.smithi120.stdout:3106 trunc from 0xdc9b983 to 0xaefcdfe 2024-09-18T01:09:26.413 INFO:teuthology.orchestra.run.smithi120.stdout:3108 writesame 0x2faa8b6 thru 0x2fb8ebc data_size 0x1b(0xe607 bytes) 2024-09-18T01:09:26.415 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:26.415 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:26.427 INFO:teuthology.orchestra.run.smithi120.stdout:3109 writesame 0xe79fc7e thru 0xe7ae72a data_size 0x1(0xeaad bytes) 2024-09-18T01:09:26.427 INFO:teuthology.orchestra.run.smithi120.stdout:3110 punch from 0xa2bfdae to 0xa2bfe0b, (0x5d bytes) 2024-09-18T01:09:26.427 INFO:teuthology.orchestra.run.smithi120.stdout:3112 read 0x930331c thru 0x9311474 (0xe159 bytes) 2024-09-18T01:09:26.427 INFO:teuthology.orchestra.run.smithi120.stdout:3113 write 0xbecea38 thru 0xbed0775 (0x1d3e bytes) 2024-09-18T01:09:26.430 INFO:teuthology.orchestra.run.smithi120.stdout:3114 writesame 0xf6948b thru 0xf6ccfa data_size 0x10(0x3870 bytes) 2024-09-18T01:09:26.431 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:26.430+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:26.432 INFO:teuthology.orchestra.run.smithi120.stdout:3116 read 0xe2e6296 thru 0xe2f4c70 (0xe9db bytes) 2024-09-18T01:09:26.432 INFO:teuthology.orchestra.run.smithi120.stdout:3117 read 0x20f91aa thru 0x2103e17 (0xac6e bytes) 2024-09-18T01:09:26.432 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.433 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:26.433 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:26.433 INFO:teuthology.orchestra.run.smithi120.stdout:3119 write 0x18459be thru 0x185485a (0xee9d bytes) 2024-09-18T01:09:26.434 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.434 INFO:teuthology.orchestra.run.smithi120.stdout:3122 write 0x466a5 thru 0x489b6 (0x2312 bytes) 2024-09-18T01:09:26.436 INFO:teuthology.orchestra.run.smithi120.stdout:3123 writesame 0xb2fa93a thru 0xb3060dc data_size 0x1(0xb7a3 bytes) 2024-09-18T01:09:26.450 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:26.450 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:26.451 INFO:teuthology.orchestra.run.smithi120.stdout:3125 punch from 0xaf1abb8 to 0xaf280c1, (0xd509 bytes) 2024-09-18T01:09:26.451 INFO:teuthology.orchestra.run.smithi120.stdout:3126 read 0x3163b98 thru 0x3166bc9 (0x3032 bytes) 2024-09-18T01:09:26.451 INFO:teuthology.orchestra.run.smithi120.stdout:3127 write 0x9ff4246 thru 0x9ff7540 (0x32fb bytes) 2024-09-18T01:09:26.467 INFO:teuthology.orchestra.run.smithi120.stdout:3128 trunc from 0xe7ae72b to 0x3507348 2024-09-18T01:09:26.468 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:26.469 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:26.487 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:26.488 INFO:teuthology.orchestra.run.smithi120.stdout:3130 write 0x521cfd7 thru 0x52270c5 (0xa0ef bytes) 2024-09-18T01:09:26.491 INFO:teuthology.orchestra.run.smithi120.stdout:3131 write 0x6b9e79 thru 0x6c628a (0xc412 bytes) 2024-09-18T01:09:26.500 INFO:teuthology.orchestra.run.smithi120.stdout:3132 trunc from 0x52270c6 to 0xab6ab36 2024-09-18T01:09:26.504 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:26.504 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:26.506 INFO:teuthology.orchestra.run.smithi120.stdout:3133 write 0x2bc9b22 thru 0x2bd935d (0xf83c bytes) 2024-09-18T01:09:26.515 INFO:teuthology.orchestra.run.smithi120.stdout:3134 trunc from 0xab6ab36 to 0x4e40b3b 2024-09-18T01:09:26.522 INFO:teuthology.orchestra.run.smithi120.stdout:3135 punch from 0x136d4a6 to 0x136efbe, (0x1b18 bytes) 2024-09-18T01:09:26.523 INFO:teuthology.orchestra.run.smithi120.stdout:3136 trunc from 0x4e40b3b to 0x65d0dd4 2024-09-18T01:09:26.524 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:26.524 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:26.530 INFO:teuthology.orchestra.run.smithi120.stdout:3137 punch from 0x4d4729 to 0x4d538d, (0xc64 bytes) 2024-09-18T01:09:26.530 INFO:teuthology.orchestra.run.smithi120.stdout:3138 read 0x51f25cf thru 0x5200344 (0xdd76 bytes) 2024-09-18T01:09:26.531 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:26.530+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:26.534 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:26.546 INFO:teuthology.orchestra.run.smithi120.stdout:3140 write 0xde37a70 thru 0xde44218 (0xc7a9 bytes) 2024-09-18T01:09:26.549 INFO:teuthology.orchestra.run.smithi120.stdout:3142 write 0x6c30b0a thru 0x6c36e0a (0x6301 bytes) 2024-09-18T01:09:26.552 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.552 INFO:teuthology.orchestra.run.smithi120.stdout:3145 read 0xb96beb1 thru 0xb97a7a6 (0xe8f6 bytes) 2024-09-18T01:09:26.552 INFO:teuthology.orchestra.run.smithi120.stdout:3146 read 0x2e21ffd thru 0x2e3149f (0xf4a3 bytes) 2024-09-18T01:09:26.553 INFO:teuthology.orchestra.run.smithi120.stdout:3148 write 0x6e04d1 thru 0x6e51c4 (0x4cf4 bytes) 2024-09-18T01:09:26.554 INFO:teuthology.orchestra.run.smithi120.stdout:3150 read 0x8c10c85 thru 0x8c1c357 (0xb6d3 bytes) 2024-09-18T01:09:26.554 INFO:teuthology.orchestra.run.smithi120.stdout:3151 read 0xd731504 thru 0xd7387ba (0x72b7 bytes) 2024-09-18T01:09:26.554 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.555 INFO:teuthology.orchestra.run.smithi120.stdout:3158 read 0x92f8cfc thru 0x9305ee2 (0xd1e7 bytes) 2024-09-18T01:09:26.555 INFO:teuthology.orchestra.run.smithi120.stdout:3159 punch from 0xcb23041 to 0xcb2cf02, (0x9ec1 bytes) 2024-09-18T01:09:26.555 INFO:teuthology.orchestra.run.smithi120.stdout:3160 trunc from 0xde44219 to 0xdaffa58 2024-09-18T01:09:26.560 INFO:teuthology.orchestra.run.smithi120.stdout:3161 writesame 0xd5e4efa thru 0xd5e6798 data_size 0xb(0x189f bytes) 2024-09-18T01:09:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:3162 read 0xa3fb618 thru 0xa3fdeb5 (0x289e bytes) 2024-09-18T01:09:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:3163 writesame 0x275db29 thru 0x2762983 data_size 0x1(0x4e5b bytes) 2024-09-18T01:09:26.575 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:09:26.575 INFO:teuthology.orchestra.run.smithi120.stdout:3164 writesame 0xb62c90e thru 0xb632dc9 data_size 0x15(0x64bc bytes) 2024-09-18T01:09:26.578 INFO:teuthology.orchestra.run.smithi120.stdout:3165 write 0x3d3dd7c thru 0x3d49e88 (0xc10d bytes) 2024-09-18T01:09:26.581 INFO:teuthology.orchestra.run.smithi120.stdout:3166 punch from 0x57b4f6e to 0x57bad84, (0x5e16 bytes) 2024-09-18T01:09:26.582 INFO:teuthology.orchestra.run.smithi120.stdout:3167 write 0xb661307 thru 0xb662639 (0x1333 bytes) 2024-09-18T01:09:26.583 INFO:teuthology.orchestra.run.smithi120.stdout:3168 write 0xa2d34e4 thru 0xa2dbf77 (0x8a94 bytes) 2024-09-18T01:09:26.586 INFO:teuthology.orchestra.run.smithi120.stdout:3170 write 0x27c2cbe thru 0x27c62ba (0x35fd bytes) 2024-09-18T01:09:26.588 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.588 INFO:teuthology.orchestra.run.smithi120.stdout:3172 writesame 0x441b83 thru 0x44fa7d data_size 0x1(0xdefb bytes) 2024-09-18T01:09:26.618 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.618 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:moved 3 / 82 (3.65854%) 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:avg 11.7143 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 2.81396 -> 2.4908 (expected baseline 3.16872) 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.6 with 7 -> 7 pgs (0.597561 -> 0.597561 * mean) 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 16 -> 15 pgs (1.36585 -> 1.28049 * mean) 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 3 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0012 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0013 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 weight 1.0000 -> 0.8601 2024-09-18T01:09:26.621 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 weight 1.0000 -> 0.9037 2024-09-18T01:09:26.622 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 weight 1.0000 -> 0.7703 2024-09-18T01:09:26.633 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:26.632+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:26.635 INFO:teuthology.orchestra.run.smithi120.stdout:3176 trunc from 0xdaffa58 to 0xabbd71 2024-09-18T01:09:26.643 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:26.643 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:26.644 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:09:26.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-18T01:09:26.653 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:26.737 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:26.736+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:26.797 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:26.798 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:26.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:26.835 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:26.834+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:26.935 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:26.934+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:26.950 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:26.950 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:26.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:27.044 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:27.043+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:27.103 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.103 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.112 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:27.137 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:27.136+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:27.238 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:27.237+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:27.256 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.256 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:27.338 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:27.337+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:27.410 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.410 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.419 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:27.464 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:27.463+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:27.523 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.523 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:27.573 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:27.572+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:27.620 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:09:27.625 INFO:teuthology.orchestra.run.smithi120.stdout:moved 3 / 82 (3.65854%) 2024-09-18T01:09:27.625 INFO:teuthology.orchestra.run.smithi120.stdout:avg 11.7143 2024-09-18T01:09:27.625 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 2.4908 -> 2.18529 (expected baseline 3.16872) 2024-09-18T01:09:27.625 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.6 with 7 -> 8 pgs (0.597561 -> 0.682927 * mean) 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.7 with 15 -> 14 pgs (1.28049 -> 1.19512 * mean) 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:oload 110 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 3 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0012 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0013 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 weight 0.8601 -> 0.7401 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 weight 0.9037 -> 0.8171 2024-09-18T01:09:27.626 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 weight 0.7703 -> 0.5923 2024-09-18T01:09:27.641 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T01:09:27.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-18T01:09:27.642 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:27.640+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:27.679 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.679 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:27.741 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:27.740+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:27.793 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.793 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.802 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:27.846 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:27.845+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:27.907 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:27.907 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:27.916 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:27.943 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:27.942+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:27.968 INFO:teuthology.orchestra.run.smithi120.stdout:moved 0 / 82 (0%) 2024-09-18T01:09:27.968 INFO:teuthology.orchestra.run.smithi120.stdout:avg 11.7143 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:stddev 2.18529 -> 2.18529 (expected baseline 3.16872) 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:min osd.6 with 8 -> 8 pgs (0.682927 -> 0.682927 * mean) 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:max osd.0 with 14 -> 14 pgs (1.19512 -> 1.19512 * mean) 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:oload 1000 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:max_change 0.05 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:max_change_osds 4 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:average_utilization 0.0012 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stdout:overload_utilization 0.0119 2024-09-18T01:09:27.969 INFO:teuthology.orchestra.run.smithi120.stderr:no change 2024-09-18T01:09:28.044 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:28.044+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:28.059 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:28.059 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:28.067 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:28.145 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:28.144+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:28.170 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:28.170 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:28.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:28.246 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:28.245+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:28.282 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:28.282 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:28.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:28.376 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:28.375+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:28.435 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:28.435 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:28.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:28.448 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:28.447+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:28.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:28.573 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:28.572+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:28.590 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:28.590 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:28.600 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:28.650 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:28.649+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:28.652 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:28.653 INFO:teuthology.orchestra.run.smithi120.stdout:3179 trunc from 0xabbd71 to 0x8edde8 2024-09-18T01:09:28.661 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:28.662 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:28.684 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:28.703 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:28.703 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:28.723 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:28.724 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:28.743 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:28.743 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:28.751 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:28.750+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:28.763 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:28.763 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:28.765 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:28.765 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:28.765 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:28.765 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:28.776 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:28.783 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:28.783 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:28.803 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:28.803 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:28.852 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:28.851+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:28.897 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:28.897 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:28.897 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:28.897 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:28.897 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:28.906 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:28.953 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:28.952+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:29.011 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4232.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 130.07022306499999, 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041778789999999998, 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.012 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939725+0000", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939743+0000", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7503000000000002e-05 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939759+0000", 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-09-18T01:09:29.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939774+0000", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4426e-05 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939831+0000", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7321000000000002e-05 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.939857+0000", 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6139000000000002e-05 2024-09-18T01:09:29.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943852+0000", 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039945730000000004 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:18.943903+0000", 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1014000000000003e-05 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2237 3.4 e28/23 3:26053817:::rbd_data.11769efc726f.0000000000000043:head v 28'8, mlcod=26'5)", 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.802027749, 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068811519999999998, 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207920+0000", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207921+0000", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208208+0000", 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028655500000000002 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208224+0000", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6478999999999999e-05 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208232+0000", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2500000000000006e-06 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208284+0000", 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2006999999999999e-05 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.208305+0000", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0268e-05 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214750+0000", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064449579999999998 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.214801+0000", 2024-09-18T01:09:29.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1767000000000001e-05 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2312 3.4 e28/23 3:263c4e2b:::rbd_data.11769efc726f.000000000000004c:head v 28'9, mlcod=26'5)", 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.800432001, 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056539379999999998, 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209516+0000", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209517+0000", 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:09:29.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209585+0000", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8620000000000004e-05 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209600+0000", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4598e-05 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209607+0000", 2024-09-18T01:09:29.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3860000000000001e-06 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209656+0000", 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9035000000000001e-05 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.209676+0000", 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9715999999999999e-05 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215127+0000", 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005451305 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215170+0000", 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2420999999999999e-05 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:2320 3.4 e28/23 3:25cbd531:::rbd_data.11769efc726f.000000000000004d:head v 28'10, mlcod=26'5)", 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:29.025 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.799621382, 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048929450000000001, 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210327+0000", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0800000000000003e-07 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210547+0000", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0002197 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210564+0000", 2024-09-18T01:09:29.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6515e-05 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210572+0000", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1149999999999994e-06 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211444+0000", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00087177100000000003 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211464+0000", 2024-09-18T01:09:29.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9987999999999999e-05 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215187+0000", 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037230969999999999 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.215220+0000", 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2851000000000001e-05 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 38/23 rep_tid 259 entries 38'30 (0'0) error 3:9390bb7a:::rbd_data.117611b60058.0000000000000156:head by client.4470.0:13250 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'28)", 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.834887428, 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046517010000000003, 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175061+0000", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.82e-07 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175068+0000", 2024-09-18T01:09:29.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175083+0000", 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5184999999999999e-05 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175091+0000", 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6819999999999992e-06 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.175142+0000", 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0921000000000003e-05 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.179712+0000", 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045699119999999998 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.033 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.4 epoch 38/23 rep_tid 163 entries 38'46 (0'0) error 3:226fc0b6:::rbd_data.117611b60058.0000000000000165:head by client.4470.0:13257 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 38'44)", 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.832230725, 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046839220000000001, 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177717+0000", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177718+0000", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3799999999999997e-07 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177726+0000", 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9440000000000005e-06 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177752+0000", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6307000000000001e-05 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177764+0000", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2003e-05 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.177840+0000", 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6243999999999997e-05 2024-09-18T01:09:29.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:45.182401+0000", 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045607859999999998 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18407 3.4 e47/23 3:24a089d4:::rbd_header.117650cf82f9:head v 47'76, mlcod=46'74)", 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.249583181999995, 2024-09-18T01:09:29.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049339570000000001, 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760365+0000", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760369+0000", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1729999999999997e-06 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760377+0000", 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3320000000000003e-06 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760382+0000", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.339e-06 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760427+0000", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.760449+0000", 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2408e-05 2024-09-18T01:09:29.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765259+0000", 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048099479999999997 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:53.765299+0000", 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9548999999999997e-05 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:18628 3.0 e47/23 3:05d85f38:::rbd_data.11762d95e74a.000000000000004a:head v 47'83, mlcod=40'80)", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.835269722999996, 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041084800000000003, 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174678+0000", 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174679+0000", 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9100000000000004e-07 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174689+0000", 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174696+0000", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9090000000000003e-06 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174701+0000", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.152e-06 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174750+0000", 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9262999999999999e-05 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.174765+0000", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4978e-05 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178753+0000", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039877070000000001 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:54.178787+0000", 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3674000000000003e-05 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39148 3.0 e63/23 3:0781abca:::rbd_data.11762bc2f01c.0000000000000013:head v 63'116, mlcod=61'114)", 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.333570700999999, 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044935340000000004, 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:29.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676377+0000", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676378+0000", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676447+0000", 2024-09-18T01:09:29.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9116999999999996e-05 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676458+0000", 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0560999999999999e-05 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676465+0000", 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0480000000000001e-06 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676519+0000", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4218999999999997e-05 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.676540+0000", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481e-05 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680822+0000", 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042816800000000004 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.680871+0000", 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8829000000000003e-05 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39150 3.0 e63/23 3:057f5581:::rbd_data.11762bc2f01c.0000000000000015:head v 63'117, mlcod=61'114)", 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.331993814, 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068572420000000004, 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677954+0000", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.677955+0000", 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.06e-07 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678098+0000", 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014364400000000001 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678112+0000", 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.346e-05 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678119+0000", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678171+0000", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1382999999999999e-05 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.678194+0000", 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3438000000000002e-05 2024-09-18T01:09:29.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684768+0000", 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065739049999999997 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.684811+0000", 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3166e-05 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.054 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:39366 3.9 e63/23 3:94b797e6:::rbd_data.11762bc2f01c.000000000000001e:head v 63'46, mlcod=62'44)", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.302246224000001, 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059816000000000001, 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707702+0000", 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707703+0000", 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707939+0000", 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023609600000000001 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707957+0000", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8034e-05 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.707967+0000", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708029+0000", 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2754000000000003e-05 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.708054+0000", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4944e-05 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713639+0000", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055845269999999997 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.713683+0000", 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4437999999999997e-05 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40765 3.0 e63/23 3:01251681:::rbd_data.11762bc2f01c.0000000000000067:head v 63'119, mlcod=63'117)", 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.146866738, 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056241809999999998, 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863081+0000", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863082+0000", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.158e-06 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863169+0000", 2024-09-18T01:09:29.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7070000000000005e-05 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863185+0000", 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5438e-05 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863192+0000", 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6869999999999997e-06 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863251+0000", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9001e-05 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.863274+0000", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3668000000000001e-05 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868657+0000", 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053832489999999997 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.868705+0000", 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.791e-05 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:40771 3.9 e63/23 3:95802560:::rbd_data.11762bc2f01c.0000000000000069:head v 63'50, mlcod=63'48)", 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.143213308, 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053147460000000004, 2024-09-18T01:09:29.063 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866735+0000", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866960+0000", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022461900000000001 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866971+0000", 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.111e-05 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.866980+0000", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867037+0000", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7043000000000003e-05 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.867053+0000", 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5773999999999998e-05 2024-09-18T01:09:29.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872005+0000", 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049527249999999998 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:10.872049+0000", 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4073999999999998e-05 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.067 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:57703 3.4 e92/23 3:25c6581d:::rbd_data.11767c61007c.0000000000000000:head v 92'144, mlcod=90'142)", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.427546632999999, 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073071220000000001, 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582401+0000", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.582402+0000", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8700000000000002e-07 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583772+0000", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001370593 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583788+0000", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5451000000000002e-05 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583797+0000", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6029999999999994e-06 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583858+0000", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0529999999999998e-05 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.583880+0000", 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2391999999999998e-05 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589674+0000", 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057937589999999999 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:39.589708+0000", 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4507e-05 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 866 entries 100'101 (0'0) error 3:9153dd03:::rbd_data.117649d3c4d0.0000000000000093:head by client.4470.0:61161 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'99)", 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.58132346, 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048029769999999999, 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:29.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428625+0000", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428633+0000", 2024-09-18T01:09:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7910000000000005e-06 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428653+0000", 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109999999999999e-05 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428667+0000", 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4187e-05 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.428764+0000", 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.433427+0000", 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046639020000000002 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 869 entries 100'103 (0'0) error 3:94e57bb7:::rbd_data.117649d3c4d0.00000000000000b2:head by client.4470.0:61192 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.568036681999999, 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047167169999999996, 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441911+0000", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441912+0000", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441920+0000", 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2260000000000005e-06 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441942+0000", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1804e-05 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.441957+0000", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4599e-05 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.442024+0000", 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7860000000000004e-05 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.446628+0000", 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046035479999999998 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(3.9 epoch 100/23 rep_tid 871 entries 100'104 (0'0) error 3:90b555ad:::rbd_data.117649d3c4d0.00000000000000ba:head by client.4470.0:61200 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 100'101)", 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.565509763000001, 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049806420000000004, 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444438+0000", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444439+0000", 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444445+0000", 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444459+0000", 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4241e-05 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444468+0000", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5399999999999996e-06 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.444541+0000", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:48.449419+0000", 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048779060000000004 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116214 3.4 e110/23 3:2420fb28:::rbd_data.11766701b08b.0000000000000277:head v 110'180, mlcod=108'178)", 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.954860409999998, 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045364760000000002, 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055088+0000", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999997e-07 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055097+0000", 2024-09-18T01:09:29.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.208e-06 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055111+0000", 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545999999999999e-05 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055118+0000", 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.934e-06 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055175+0000", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7240000000000001e-05 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.055201+0000", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5978000000000001e-05 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059539+0000", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043376889999999996 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.059624+0000", 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5118999999999996e-05 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116558 3.9 e110/23 3:9119117f:::rbd_data.11766701b08b.0000000000000127:head v 110'143, mlcod=110'141)", 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.802042434000001, 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005246901, 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207906+0000", 2024-09-18T01:09:29.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207914+0000", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9929999999999995e-06 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207925+0000", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0828e-05 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207934+0000", 2024-09-18T01:09:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0860000000000008e-06 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207976+0000", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2274000000000002e-05 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.207991+0000", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4296e-05 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213113+0000", 2024-09-18T01:09:29.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051225649999999999 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.213152+0000", 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9116999999999998e-05 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:116699 3.4 e110/23 3:27f7a2f6:::rbd_data.11766701b08b.00000000000001bf:head v 110'196, mlcod=110'193)", 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.743420039, 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042393159999999999, 2024-09-18T01:09:29.091 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266528+0000", 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266529+0000", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266534+0000", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5500000000000002e-06 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266545+0000", 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0926e-05 2024-09-18T01:09:29.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266551+0000", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3049999999999998e-06 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266596+0000", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4864999999999998e-05 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.266617+0000", 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0682e-05 2024-09-18T01:09:29.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270721+0000", 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041039140000000002 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:58.270767+0000", 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.634e-05 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.095 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:29.097 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:29.097 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:29.096+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:29.197 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:29.197+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:29.239 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:29.239 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:29.239 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:29.240 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:29.248 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:29.298 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:29.297+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:29.351 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:29.351 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:29.351 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:29.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:29.351 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:29.359 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:29.403 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:29.402+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833413637, 2024-09-18T01:09:29.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:29.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833388651999996, 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.468 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:29.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833368872999998, 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833342471000002, 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.477 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:29.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833320974999999, 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833301679000002, 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:29.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:29.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833283844999997, 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:29.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:29.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:29.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833257781999997, 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.499 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833238426999998, 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833208182999996, 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:29.504 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.508 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833189588000003, 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:29.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833171028999999, 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:29.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833152974999997, 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.519 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:29.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:29.523 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833135216000002, 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:29.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:29.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833108946999999, 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:29.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833090327000001, 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.833072076999997, 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.832520701999997, 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.830921715999999, 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.827155728999998, 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:29.555 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:29.556 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:29.555+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:29.557 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:29.588 INFO:teuthology.orchestra.run.smithi120.stdout:3181 read 0xed4fd thru 0xfaa0b (0xd50f bytes) 2024-09-18T01:09:29.588 INFO:teuthology.orchestra.run.smithi120.stdout:3182 punch from 0x1d52b2 to 0x1e3c69, (0xe9b7 bytes) 2024-09-18T01:09:29.589 INFO:teuthology.orchestra.run.smithi120.stdout:3183 trunc from 0x8edde8 to 0x519728 2024-09-18T01:09:29.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:09:29.657 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:29.656+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:29.671 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:29.671 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:29.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:29.764 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:29.763+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:29.826 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:29.826 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:29.826 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:29.826 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:29.826 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:29.836 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:29.858 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:29.857+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:29.959 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:29.958+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:29.988 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:29.988 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:29.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:29.988 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.321947642, 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:29.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:09:29.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:09:29.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:09:29.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.263760063000007, 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:29.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:29.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:29.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:29.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124264459999999, 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:29.999 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:09:30.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124241941000001, 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:09:30.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:30.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124220373, 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:09:30.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:30.012 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124199603999998, 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.016 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124180408999997, 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.017 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:09:30.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:09:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124161622999999, 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:09:30.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.025 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124142313999997, 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:09:30.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124119925000002, 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.030 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:09:30.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:09:30.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124098605, 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.035 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:09:30.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124075011999999, 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:09:30.039 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:09:30.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124055095999999, 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.044 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:09:30.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:09:30.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124036664999998, 2024-09-18T01:09:30.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:09:30.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.124012401000002, 2024-09-18T01:09:30.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:30.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:09:30.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.123988027000003, 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:09:30.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.123966887000002, 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:30.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:09:30.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.123946805000003, 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:30.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:09:30.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.123926658999999, 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:09:30.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:09:30.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.123906595999998, 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.081 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.083 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:30.083 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:30.081+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:30.183 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:30.182+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:30.228 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:30.228 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:30.284 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:30.283+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:30.379 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:30.379 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:30.379 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:30.379 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:30.380 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.385 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:30.384+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:30.388 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:30.486 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:30.485+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467419819, 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:30.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:30.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467377668, 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.497 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:30.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.46734424, 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:30.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:30.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.46730674, 2024-09-18T01:09:30.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:30.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467285163, 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:30.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:30.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.46726602299999, 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:30.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467245632, 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:30.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:30.525 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467216232, 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:30.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467187442, 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.530 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:30.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.46716674699999, 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:30.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467147263, 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:30.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.467113352, 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:30.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:30.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:30.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.46707735299999, 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:30.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.382629381, 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:30.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.38261917200001, 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:30.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:30.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.882217946, 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:30.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:30.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:30.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.866731777, 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:30.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:30.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.832772054, 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:30.572 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:30.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:30.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:30.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.578 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.827603262, 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:121600 3.5 3:a5560957:::rbd_header.1176e38d32d8:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.9203604040000002, 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034695009999999998, 2024-09-18T01:09:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 121600 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570147+0000", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.85e-07 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570151+0000", 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570162+0000", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0317e-05 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570167+0000", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570207+0000", 2024-09-18T01:09:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0543999999999997e-05 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570262+0000", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4539000000000001e-05 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570403+0000", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140846 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570909+0000", 2024-09-18T01:09:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050672900000000003 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604784+0000", 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033874597999999999 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604816+0000", 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1668000000000002e-05 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604841+0000", 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999999e-05 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:30.589 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.590 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:09:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:3184 read 0x3dc935 thru 0x3e7f41 (0xb60d bytes) 2024-09-18T01:09:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:3185 read 0x318f36 thru 0x325e26 (0xcef1 bytes) 2024-09-18T01:09:30.591 INFO:teuthology.orchestra.run.smithi120.stdout:3186 read 0x27ca62 thru 0x28b003 (0xe5a2 bytes) 2024-09-18T01:09:30.591 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:30.592 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:30.589+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:30.691 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:30.690+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:30.735 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:30.736 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.745 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:30.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:30.829 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:30.828+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:30.888 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:30.888 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:30.888 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:30.888 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:30.889 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:30.893 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:30.892+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:30.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:30.936 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:30.936 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:30.953 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:30.953 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:30.969 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:30.969 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:30.985 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:30.985 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:30.993 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:30.992+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:31.003 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:31.003 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:31.020 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:31.020 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:31.036 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:31.037 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:31.042 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.042 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:31.053 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:31.053 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:31.095 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:31.094+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:31.154 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.154 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.163 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:31.204 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:31.203+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:31.295 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:31.294+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:31.305 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.305 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.313 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:31.396 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:31.395+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:31.416 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.416 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.424 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:31.497 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:31.496+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:31.527 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.528 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.536 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:31.576 INFO:teuthology.orchestra.run.smithi120.stdout:3188 write 0x10e8028 thru 0x10f359b (0xb574 bytes) 2024-09-18T01:09:31.596 INFO:teuthology.orchestra.run.smithi120.stdout:3189 trunc from 0x10f359c to 0xe14e89b 2024-09-18T01:09:31.598 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:31.597+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:31.602 INFO:teuthology.orchestra.run.smithi120.stdout:3190 read 0x695bc29 thru 0x6964183 (0x855b bytes) 2024-09-18T01:09:31.602 INFO:teuthology.orchestra.run.smithi120.stdout:3192 read 0xb671066 thru 0xb680849 (0xf7e4 bytes) 2024-09-18T01:09:31.602 INFO:teuthology.orchestra.run.smithi120.stdout:3193 write 0xbb7042d thru 0xbb7b8ed (0xb4c1 bytes) 2024-09-18T01:09:31.605 INFO:teuthology.orchestra.run.smithi120.stdout:3195 read 0x91f86c5 thru 0x92003a2 (0x7cde bytes) 2024-09-18T01:09:31.605 INFO:teuthology.orchestra.run.smithi120.stdout:3196 read 0x6d1171f thru 0x6d1f419 (0xdcfb bytes) 2024-09-18T01:09:31.605 INFO:teuthology.orchestra.run.smithi120.stdout:3198 write 0x4c942cc thru 0x4c9c38e (0x80c3 bytes) 2024-09-18T01:09:31.608 INFO:teuthology.orchestra.run.smithi120.stdout:3199 punch from 0x623c9e1 to 0x623d5df, (0xbfe bytes) 2024-09-18T01:09:31.608 INFO:teuthology.orchestra.run.smithi120.stdout:3200 write 0xa924a57 thru 0xa929c22 (0x51cc bytes) 2024-09-18T01:09:31.610 INFO:teuthology.orchestra.run.smithi120.stdout:3201 writesame 0x97fc293 thru 0x97fd5c1 data_size 0x1(0x132f bytes) 2024-09-18T01:09:31.614 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.614 INFO:teuthology.orchestra.run.smithi120.stdout:3203 punch from 0x4856b51 to 0x485d273, (0x6722 bytes) 2024-09-18T01:09:31.620 INFO:teuthology.orchestra.run.smithi120.stdout:3204 trunc from 0xe14e89b to 0x95730f5 2024-09-18T01:09:31.633 INFO:teuthology.orchestra.run.smithi120.stdout:3205 trunc from 0x95730f5 to 0xdce69ad 2024-09-18T01:09:31.652 INFO:teuthology.orchestra.run.smithi120.stdout:3206 trunc from 0xdce69ad to 0x2445bf3 2024-09-18T01:09:31.668 INFO:teuthology.orchestra.run.smithi120.stdout:3207 read 0x7be2a thru 0x8b889 (0xfa60 bytes) 2024-09-18T01:09:31.668 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.668 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.671 INFO:teuthology.orchestra.run.smithi120.stdout:3209 trunc from 0x2445bf3 to 0x49dbebf 2024-09-18T01:09:31.680 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:31.684 INFO:teuthology.orchestra.run.smithi120.stdout:3210 write 0x60ccba5 thru 0x60d878e (0xbbea bytes) 2024-09-18T01:09:31.687 INFO:teuthology.orchestra.run.smithi120.stdout:3213 punch from 0x173f64b to 0x174f4f6, (0xfeab bytes) 2024-09-18T01:09:31.691 INFO:teuthology.orchestra.run.smithi120.stdout:3216 trunc from 0x60d878f to 0x98769e3 2024-09-18T01:09:31.697 INFO:teuthology.orchestra.run.smithi120.stdout:3217 write 0x191f51e thru 0x1924969 (0x544c bytes) 2024-09-18T01:09:31.698 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:31.697+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:31.699 INFO:teuthology.orchestra.run.smithi120.stdout:3218 writesame 0x21466b2 thru 0x215307b data_size 0x17(0xc9ca bytes) 2024-09-18T01:09:31.701 INFO:teuthology.orchestra.run.smithi120.stdout:3219 punch from 0x86787da to 0x867aae9, (0x230f bytes) 2024-09-18T01:09:31.708 INFO:teuthology.orchestra.run.smithi120.stdout:3222 trunc from 0x98769e3 to 0x3fcbae6 2024-09-18T01:09:31.739 INFO:teuthology.orchestra.run.smithi120.stdout:3224 write 0xe34b7e7 thru 0xe358cec (0xd506 bytes) 2024-09-18T01:09:31.745 INFO:teuthology.orchestra.run.smithi120.stdout:3225 read 0x29740e1 thru 0x297e7b3 (0xa6d3 bytes) 2024-09-18T01:09:31.745 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.745 INFO:teuthology.orchestra.run.smithi120.stdout:3229 write 0x4a62043 thru 0x4a64cdb (0x2c99 bytes) 2024-09-18T01:09:31.752 INFO:teuthology.orchestra.run.smithi120.stdout:3230 trunc from 0xe358ced to 0xaf05276 2024-09-18T01:09:31.766 INFO:teuthology.orchestra.run.smithi120.stdout:3232 trunc from 0xaf05276 to 0xe484553 2024-09-18T01:09:31.773 INFO:teuthology.orchestra.run.smithi120.stdout:3234 read 0x4858c0a thru 0x4863574 (0xa96b bytes) 2024-09-18T01:09:31.773 INFO:teuthology.orchestra.run.smithi120.stdout:3235 read 0x688435b thru 0x688b2df (0x6f85 bytes) 2024-09-18T01:09:31.773 INFO:teuthology.orchestra.run.smithi120.stdout:3238 punch from 0x502011e to 0x5027033, (0x6f15 bytes) 2024-09-18T01:09:31.773 INFO:teuthology.orchestra.run.smithi120.stdout:3239 write 0xa9e70df thru 0xa9f1c5b (0xab7d bytes) 2024-09-18T01:09:31.776 INFO:teuthology.orchestra.run.smithi120.stdout:3240 punch from 0x9c8c19c to 0x9c99b44, (0xd9a8 bytes) 2024-09-18T01:09:31.776 INFO:teuthology.orchestra.run.smithi120.stdout:3241 read 0x102d307 thru 0x10349b5 (0x76af bytes) 2024-09-18T01:09:31.776 INFO:teuthology.orchestra.run.smithi120.stdout:3242 write 0x918f5b0 thru 0x919b6f7 (0xc148 bytes) 2024-09-18T01:09:31.778 INFO:teuthology.orchestra.run.smithi120.stdout:3244 writesame 0x858a50a thru 0x8595a0a data_size 0x1(0xb501 bytes) 2024-09-18T01:09:31.798 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.798 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.799 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:31.798+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:31.805 INFO:teuthology.orchestra.run.smithi120.stdout:3246 read 0xb577160 thru 0xb580c25 (0x9ac6 bytes) 2024-09-18T01:09:31.805 INFO:teuthology.orchestra.run.smithi120.stdout:3247 read 0x96dc56d thru 0x96e72bc (0xad50 bytes) 2024-09-18T01:09:31.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:31.811 INFO:teuthology.orchestra.run.smithi120.stdout:3248 trunc from 0xe484553 to 0x9be4620 2024-09-18T01:09:31.819 INFO:teuthology.orchestra.run.smithi120.stdout:3249 read 0x5a2892a thru 0x5a34aca (0xc1a1 bytes) 2024-09-18T01:09:31.820 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.822 INFO:teuthology.orchestra.run.smithi120.stdout:3251 trunc from 0x9be4620 to 0xc6ac823 2024-09-18T01:09:31.828 INFO:teuthology.orchestra.run.smithi120.stdout:3252 punch from 0x3fe6f82 to 0x3fec7d6, (0x5854 bytes) 2024-09-18T01:09:31.828 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.828 INFO:teuthology.orchestra.run.smithi120.stdout:3256 read 0x420593d thru 0x420bd72 (0x6436 bytes) 2024-09-18T01:09:31.828 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.828 INFO:teuthology.orchestra.run.smithi120.stdout:3258 write 0x3ad3c31 thru 0x3ad4ad6 (0xea6 bytes) 2024-09-18T01:09:31.831 INFO:teuthology.orchestra.run.smithi120.stdout:3259 write 0x7c08ab9 thru 0x7c11ec6 (0x940e bytes) 2024-09-18T01:09:31.834 INFO:teuthology.orchestra.run.smithi120.stdout:3261 writesame 0x91bb85f thru 0x91bd5f0 data_size 0x5(0x1d92 bytes) 2024-09-18T01:09:31.836 INFO:teuthology.orchestra.run.smithi120.stdout:3262 write 0x68dc66f thru 0x68e8aa8 (0xc43a bytes) 2024-09-18T01:09:31.846 INFO:teuthology.orchestra.run.smithi120.stdout:3264 trunc from 0xc6ac823 to 0x716761c 2024-09-18T01:09:31.866 INFO:teuthology.orchestra.run.smithi120.stdout:3265 trunc from 0x716761c to 0xeb3acc6 2024-09-18T01:09:31.872 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.872 INFO:teuthology.orchestra.run.smithi120.stdout:3270 read 0x1cbbe2b thru 0x1cc5a03 (0x9bd9 bytes) 2024-09-18T01:09:31.872 INFO:teuthology.orchestra.run.smithi120.stdout:3271 write 0x8cb043b thru 0x8cb1721 (0x12e7 bytes) 2024-09-18T01:09:31.873 INFO:teuthology.orchestra.run.smithi120.stdout:3272 read 0x642871b thru 0x642cd80 (0x4666 bytes) 2024-09-18T01:09:31.874 INFO:teuthology.orchestra.run.smithi120.stdout:3273 read 0x6c8679e thru 0x6c9516b (0xe9ce bytes) 2024-09-18T01:09:31.874 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.874 INFO:teuthology.orchestra.run.smithi120.stdout:3275 writesame 0x6efa6f5 thru 0x6efc97d data_size 0x7(0x2289 bytes) 2024-09-18T01:09:31.875 INFO:teuthology.orchestra.run.smithi120.stdout:3276 write 0xc52f7ba thru 0xc53aaa6 (0xb2ed bytes) 2024-09-18T01:09:31.878 INFO:teuthology.orchestra.run.smithi120.stdout:3277 writesame 0x629e851 thru 0x62a839b data_size 0x1(0x9b4b bytes) 2024-09-18T01:09:31.898 INFO:teuthology.orchestra.run.smithi120.stdout:3279 read 0xd3dd0b1 thru 0xd3ddfc9 (0xf19 bytes) 2024-09-18T01:09:31.899 INFO:teuthology.orchestra.run.smithi120.stdout:3280 writesame 0x3adfcb5 thru 0x3aebbac data_size 0x7e(0xbef8 bytes) 2024-09-18T01:09:31.900 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:31.899+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:31.913 INFO:teuthology.orchestra.run.smithi120.stdout:3281 trunc from 0xeb3acc6 to 0x528af11 2024-09-18T01:09:31.925 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:31.926 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:31.927 INFO:teuthology.orchestra.run.smithi120.stdout:3282 punch from 0x40f8737 to 0x4102f7d, (0xa846 bytes) 2024-09-18T01:09:31.927 INFO:teuthology.orchestra.run.smithi120.stdout:3283 writesame 0x4a09116 thru 0x4a125f0 data_size 0x1(0x94db bytes) 2024-09-18T01:09:31.935 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:31.944 INFO:teuthology.orchestra.run.smithi120.stdout:3284 read 0x2f6cda7 thru 0x2f74349 (0x75a3 bytes) 2024-09-18T01:09:31.945 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:31.945 INFO:teuthology.orchestra.run.smithi120.stdout:3286 clone 30 order 19 su 2048 sc 14 2024-09-18T01:09:32.000 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:31.999+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:32.047 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.047 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:32.101 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:32.100+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:32.158 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.158 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:32.211 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:32.210+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:32.270 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.270 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.278 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:32.320 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:32.319+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:32.404 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:32.403+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:32.423 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.423 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:32.516 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:32.515+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:32.574 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.575 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:32.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:09:32.606 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:32.605+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:32.706 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:32.705+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:32.735 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.735 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:32.808 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:32.807+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.859 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:32.908 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:32.907+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:32.963 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:32.963 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:32.972 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:32.982 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:32.983 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:32.983 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-18T01:09:32.983 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-18T01:09:33.027 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:33.026+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:33.064 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:33.088 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:33.088 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:33.098 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:33.110 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:33.109+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:33.188 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:33.189 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:33.202 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:33.202 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:33.205 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:33.205 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:33.211 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:33.212 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:33.210+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:33.222 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:33.222 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:33.238 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:33.254 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:33.254 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:33.271 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:33.280 INFO:tasks.ceph.mon.a.smithi120.stderr:2024-09-18T01:09:33.280+0000 7f77b87d3640 -1 mon.a@0(leader).osd e139 definitely_dead 0 2024-09-18T01:09:33.287 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:33.287 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:33.304 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:33.304 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:33.312 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:33.311+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.6517815, 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:09:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:09:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.593593920999993, 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.454098318, 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.454075799000002, 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:09:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:09:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.454054231000001, 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:09:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.454033461999998, 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:09:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:09:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.454014266999998, 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:09:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453995481, 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:09:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453976171999997, 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:09:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453953783000003, 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:09:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453932463000001, 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453908869999999, 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453888954, 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:09:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:09:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:09:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453870522999999, 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:09:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:09:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:09:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453846259000002, 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:09:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453821884999996, 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:09:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:09:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453800745000002, 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:09:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:09:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453780663000003, 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:09:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:09:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453760516999999, 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:09:33.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.453740453999998, 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:09:33.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:09:33.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.428 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:33.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:33.430 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:33.428+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:33.530 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:33.529+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:33.574 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:33.574 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:33.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:33.631 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:33.630+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:33.647 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:09:33.647 INFO:teuthology.orchestra.run.smithi120.stderr:marked in osd.2. 2024-09-18T01:09:33.667 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-18T01:09:33.729 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:33.729 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:33.731 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:33.730+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:33.737 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:33.832 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:33.831+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:33.842 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:33.842 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81777382999999, 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:33.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:33.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817731679, 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:33.847 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:33.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817698251, 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.852 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:33.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.856 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81766075100001, 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81763917399999, 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:33.861 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:33.865 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817620034, 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.866 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:33.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:33.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81759964299999, 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:33.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81757024300001, 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.875 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:33.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817541453, 2024-09-18T01:09:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817520758, 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.884 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:33.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81750127399999, 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:33.888 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:33.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:33.892 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.81746736300001, 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.893 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:33.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:33.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.817431364, 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:33.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:33.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:33.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.73298339199999, 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:33.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 118.732973183, 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.906 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:33.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:33.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:33.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.232571957, 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:33.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:33.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:33.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:33.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.21708578800001, 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:33.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.183126065, 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:33.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:33.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:33.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:33.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:33.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.177957273, 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:33.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:33.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:121600 3.5 3:a5560957:::rbd_header.1176e38d32d8:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.2707144150000005, 2024-09-18T01:09:33.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034695009999999998, 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 121600 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570147+0000", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.85e-07 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570151+0000", 2024-09-18T01:09:33.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570162+0000", 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0317e-05 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570167+0000", 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570207+0000", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0543999999999997e-05 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570262+0000", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4539000000000001e-05 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570403+0000", 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140846 2024-09-18T01:09:33.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570909+0000", 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050672900000000003 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604784+0000", 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033874597999999999 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:33.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604816+0000", 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1668000000000002e-05 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604841+0000", 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999999e-05 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:33.938 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:33.940 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:33.940 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:33.939+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:34.040 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:34.039+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:34.084 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.084 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:34.084 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:34.084 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:34.092 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:34.176 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:34.175+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:34.235 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.235 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:34.235 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:34.235 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:34.235 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:34.242 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:34.242+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:34.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:34.343 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:34.342+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:34.389 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.816605300999996, 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:34.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:34.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:34.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.396 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.764614158000001, 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:34.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.401 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.764537361999999, 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:34.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.760032209000002, 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.407 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:34.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.411 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.759776412999997, 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:34.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.755751085, 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:34.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:34.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.755609934999995, 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:34.422 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:34.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.427 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.755377030999995, 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:34.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.752757969000001, 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.437 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.730418935000003, 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.438 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:34.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:34.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779903660999999, 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:34.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779881920000001, 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779861511999997, 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:34.452 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:34.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779841853000001, 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:34.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:34.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:34.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779808518000003, 2024-09-18T01:09:34.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779773986000002, 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779729095999997, 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:34.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779702721, 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:34.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:34.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779683570000003, 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:34.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.779665002000002, 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:34.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.488 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:34.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:34.490 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:34.488+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:34.589 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:34.588+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:34.636 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:34.636 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:34.644 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:34.728 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:34.727+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:34.788 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:34.788 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:34.792 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:34.791+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:34.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:34.892 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:34.891+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:34.941 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 143.04205240300001, 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318535570000002, 2024-09-18T01:09:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318514159000003, 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318491244000001, 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318470226999999, 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.31843902, 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318416692, 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318394668000003, 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318345241000003, 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318314143999999, 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318291148999997, 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318269430999997, 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318248009999998, 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318226852999999, 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318201874000003, 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318170551000001, 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318147662000001, 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318124032999997, 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.318101851999998, 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3505973850000004, 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:09:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:09:35.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:35.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:35.035 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:35.034+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:35.036 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:35.136 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:35.135+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:35.152 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:35.152 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:35.152 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:35.152 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:35.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:35.236 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:35.236+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:35.305 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:35.306 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:35.306 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:35.306 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:35.306 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:35.314 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:35.316 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:35.337 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:35.336+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:35.422 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:35.422 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.82418076899999, 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:35.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.762578147, 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:35.429 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:35.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:35.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.697728759, 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:35.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:35.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:35.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456254121000001, 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:35.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456229956000001, 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:35.447 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:35.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:35.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456208848000003, 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:35.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456187974999999, 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.457 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:35.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456169748999997, 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:35.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456077579000002, 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:35.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456049168, 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:35.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456027177000003, 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:35.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.456007196000002, 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:35.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.455986338999999, 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451210699000001, 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:35.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451146883, 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:35.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451121639, 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.502 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451082892000002, 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:35.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451060577, 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.507 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:35.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451038066999999, 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:35.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:35.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.451001757999997, 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:35.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:35.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:35.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:35.520 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:35.522 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:35.522 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:35.522 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:35.523 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:35.523 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:35.523 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:35.523 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:35.524 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:35.524 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:35.524 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:35.525 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:35.525 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:35.525 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:35.526 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:35.520+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:35.535 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:35.535 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:35.552 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:35.552 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:35.622 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:35.621+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:35.668 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:35.669 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:35.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:09:35.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:35.722 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:35.721+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:35.823 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:35.822+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:35.877 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:35.877 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:35.889 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:35.923 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:35.922+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:36.024 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:36.023+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:36.067 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:36.067 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:36.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:36.125 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:36.124+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:36.226 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:36.225+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:36.303 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:36.304 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:36.317 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:36.362 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:36.361+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:36.428 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:36.427+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:36.491 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:36.491 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:36.505 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:36.546 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:36.545+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:36.630 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:36.629+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:36.661 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:09:36.704 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:36.705 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:36.715 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:36.730 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:36.730+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:36.832 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:36.831+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:36.859 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:36.859 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:36.867 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:36.952 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:36.951+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:37.012 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.012 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:37.033 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:37.032+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:37.134 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:37.133+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:37.164 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.164 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.172 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:37.235 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:37.234+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:37.277 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.277 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.285 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:37.370 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:37.369+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:37.429 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.436 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:37.436+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:37.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:37.537 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:37.536+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:37.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:37.581 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.581 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:37.674 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:37.673+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:37.702 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:37.703 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:37.727 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:37.727 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:37.739 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:37.738+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:37.752 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:37.752 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:37.753 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:37.753 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:37.764 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:37.779 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:37.780 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:37.805 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:37.831 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:37.831 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:37.840 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:37.839+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:37.852 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:37.852 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:37.871 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:37.871 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:37.927 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:37.928 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:37.937 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:37.981 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:37.980+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016937349, 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016895198, 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.01686177000001, 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.01682427, 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016802693, 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016783553, 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016763162, 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016733762, 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016704972, 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016684277, 2024-09-18T01:09:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016664793, 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016630882, 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.016594883, 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.932146911, 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.93213670199999, 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 117.431735476, 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:38.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 117.416249307, 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.117 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 117.38228958400001, 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.121 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:38.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:38.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 117.377120792, 2024-09-18T01:09:38.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:38.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:121600 3.5 3:a5560957:::rbd_header.1176e38d32d8:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.469877933999999, 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034695009999999998, 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 121600 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570147+0000", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.85e-07 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570151+0000", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570162+0000", 2024-09-18T01:09:38.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0317e-05 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570167+0000", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570207+0000", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0543999999999997e-05 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570262+0000", 2024-09-18T01:09:38.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4539000000000001e-05 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570403+0000", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140846 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570909+0000", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050672900000000003 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604784+0000", 2024-09-18T01:09:38.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033874597999999999 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604816+0000", 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1668000000000002e-05 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604841+0000", 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999999e-05 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.138 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.138 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:38.040+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:38.139 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:38.180 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:38.179+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:38.243 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:38.242+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:38.281 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.281 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:38.281 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:38.281 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.290 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:38.344 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:38.343+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.393 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:38.393 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:38.393 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.402 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:38.445 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:38.444+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:38.506 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.933821657999999, 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.881830515000004, 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.881753719000002, 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.877248566000006, 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.876992770000001, 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.872967442000004, 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.872826291999999, 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.872593387999999, 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.869974326000005, 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.847635292000007, 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.897120018000003, 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.897098276999998, 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.897077869, 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.897058209999997, 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.897024875, 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.896990342999999, 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:38.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.896945453000001, 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.896919078000003, 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:38.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.896899927, 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.597 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:38.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:38.601 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.896881358999998, 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:38.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:38.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:38.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:38.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:38.606 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:38.608 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:38.606+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:38.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:38.668 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:38.668 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:09:38.668 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:09:38.679 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:09:38.708 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:38.707+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:38.757 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-18T01:09:38.757 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.758 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:38.758 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:38.758 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.767 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:38.810 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:38.809+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:38.836 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:09:38.910 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:38.909+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:38.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:38.913 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:38.921 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:38.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:09:39.012 INFO:teuthology.orchestra.run.smithi120.stdout:3287 write 0xeb148fe thru 0xeb18c59 (0x435c bytes) 2024-09-18T01:09:39.013 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:39.012+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:39.045 INFO:teuthology.orchestra.run.smithi120.stdout:3288 trunc from 0xeb18c5a to 0x207263c 2024-09-18T01:09:39.080 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 147.18048207199999, 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456965238999999, 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456943828, 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456920912999998, 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456899896000003, 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456868688999997, 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456846360999997, 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456824337, 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.45677491, 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456743813000003, 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456720818000001, 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456699100000002, 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:39.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456677679000002, 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.137 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:39.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:39.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:39.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456656522000003, 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:39.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456631543, 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:39.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:39.150 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456600219999999, 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456577330999998, 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:39.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:39.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:39.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:39.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456553702000001, 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:39.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.456531521000002, 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:39.163 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:39.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:39.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.4890270539999992, 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:39.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:39.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:09:39.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:09:39.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:39.176 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:39.177 INFO:teuthology.orchestra.run.smithi120.stdout:3289 punch from 0x1ee0f42 to 0x1ee717e, (0x623c bytes) 2024-09-18T01:09:39.177 INFO:teuthology.orchestra.run.smithi120.stdout:3290 write 0x54c5e27 thru 0x54c9ba6 (0x3d80 bytes) 2024-09-18T01:09:39.178 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:39.176+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:39.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:39.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:09:39.180 INFO:teuthology.orchestra.run.smithi120.stdout:3291 writesame 0x49265bb thru 0x492e6d7 data_size 0x1(0x811d bytes) 2024-09-18T01:09:39.180 INFO:teuthology.orchestra.run.smithi120.stdout:3292 read 0x52d0e8a thru 0x52dbc1c (0xad93 bytes) 2024-09-18T01:09:39.180 INFO:teuthology.orchestra.run.smithi120.stdout:3294 write 0xe1fea9e thru 0xe20d47b (0xe9de bytes) 2024-09-18T01:09:39.191 INFO:teuthology.orchestra.run.smithi120.stdout:3296 write 0x8cd26c1 thru 0x8cdd944 (0xb284 bytes) 2024-09-18T01:09:39.206 INFO:teuthology.orchestra.run.smithi120.stdout:3297 read 0x63f6dbf thru 0x63f923d (0x247f bytes) 2024-09-18T01:09:39.206 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.206 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.206 INFO:teuthology.orchestra.run.smithi120.stdout:3300 punch from 0xc02571b to 0xc027bf2, (0x24d7 bytes) 2024-09-18T01:09:39.212 INFO:teuthology.orchestra.run.smithi120.stdout:3301 write 0xea71c1d thru 0xea78569 (0x694d bytes) 2024-09-18T01:09:39.227 INFO:teuthology.orchestra.run.smithi120.stdout:3303 writesame 0x2b21527 thru 0x2b2c352 data_size 0x4(0xae2c bytes) 2024-09-18T01:09:39.254 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.254 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.254 INFO:teuthology.orchestra.run.smithi120.stdout:3307 punch from 0x11da313 to 0x11e9faf, (0xfc9c bytes) 2024-09-18T01:09:39.254 INFO:teuthology.orchestra.run.smithi120.stdout:3308 punch from 0x53fa881 to 0x53ff930, (0x50af bytes) 2024-09-18T01:09:39.254 INFO:teuthology.orchestra.run.smithi120.stdout:3309 write 0xe097a70 thru 0xe09d218 (0x57a9 bytes) 2024-09-18T01:09:39.274 INFO:teuthology.orchestra.run.smithi120.stdout:3310 trunc from 0xea7856a to 0x149c683 2024-09-18T01:09:39.278 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:39.277+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144176 3.12 3:4e975413:::rbd_data.117650babc70.00000000000001d4:head [delete] snapc 3f=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.319898+0000", 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.001699037, 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017172470000000001, 2024-09-18T01:09:39.322 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144176 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319898+0000", 2024-09-18T01:09:39.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319898+0000", 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319899+0000", 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5820000000000001e-06 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319910+0000", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0509000000000001e-05 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319924+0000", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4763000000000001e-05 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.319933+0000", 2024-09-18T01:09:39.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0289999999999994e-06 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320004+0000", 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0819000000000001e-05 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320096+0000", 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1705000000000004e-05 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320209+0000", 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113478 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.321247+0000", 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001038017 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.327 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:144177 3.18 e145/23 3:185e46e5:::rbd_data.117650babc70.00000000000001d5:head v 145'309, mlcod=145'307)", 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.320503+0000", 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0010931980000000001, 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011796269999999999, 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320503+0000", 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320503+0000", 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320504+0000", 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7900000000000007e-07 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320511+0000", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1310000000000003e-06 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320523+0000", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2792000000000001e-05 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320530+0000", 2024-09-18T01:09:39.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5220000000000004e-06 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320708+0000", 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017840299999999999 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320737+0000", 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8958999999999998e-05 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:144175 3.10 e145/82 3:0c289fae:::rbd_data.117650babc70.00000000000001d3:head v 145'361, mlcod=145'359)", 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.320909+0000", 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.000687131, 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00080563100000000001, 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320909+0000", 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320909+0000", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320910+0000", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.99e-07 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320916+0000", 2024-09-18T01:09:39.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5470000000000004e-06 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320924+0000", 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5320000000000006e-06 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.320930+0000", 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8989999999999998e-06 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.321063+0000", 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132967 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.321081+0000", 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7887999999999999e-05 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:09:39.335 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 3 2024-09-18T01:09:39.336 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:39.336 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:39.340 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.340 INFO:teuthology.orchestra.run.smithi120.stdout:3314 punch from 0x8cc195 to 0x8cce62, (0xccd bytes) 2024-09-18T01:09:39.340 INFO:teuthology.orchestra.run.smithi120.stdout:3316 read 0xa8aa17 thru 0xa97246 (0xc830 bytes) 2024-09-18T01:09:39.340 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.354 INFO:teuthology.orchestra.run.smithi120.stdout:3319 writesame 0x76af12f thru 0x76bd884 data_size 0x1(0xe756 bytes) 2024-09-18T01:09:39.354 INFO:teuthology.orchestra.run.smithi120.stdout:3320 write 0x3ca5df3 thru 0x3ca61d3 (0x3e1 bytes) 2024-09-18T01:09:39.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T01:09:39.366 INFO:teuthology.orchestra.run.smithi120.stdout:3322 write 0x8f58634 thru 0x8f64be8 (0xc5b5 bytes) 2024-09-18T01:09:39.378 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:39.377+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:39.381 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:39.381 INFO:teuthology.orchestra.run.smithi120.stdout:3326 read 0x48045ef thru 0x480ea45 (0xa457 bytes) 2024-09-18T01:09:39.381 INFO:teuthology.orchestra.run.smithi120.stdout:3327 clone 31 order 20 su 4096 sc 8 2024-09-18T01:09:39.450 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:39.450 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:39.450 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:39.450 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:39.451 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:39.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:39.501 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:39.500+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:39.503 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T01:09:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:09:39.616 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.018602978, 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:39.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:39.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:39.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:39.623 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.95700035599999, 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:39.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:39.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.892150968, 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:39.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:39.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650676330000003, 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.641 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650652164999997, 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:39.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650631056999998, 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650610184000001, 2024-09-18T01:09:39.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:39.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:39.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650591958, 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:39.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:39.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650499787999998, 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:39.659 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:39.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:39.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650471377000002, 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:39.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:39.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:39.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650449385999998, 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:39.672 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650429404999997, 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:39.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.650408548000001, 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.677 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:39.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:39.681 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645632908000003, 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:39.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:39.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:39.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645569092000002, 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645543848000003, 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:39.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645505100999998, 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.695 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:39.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:39.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645482786000002, 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:39.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645460276000001, 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:39.703 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.645423966999999, 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:39.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:39.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:39.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:39.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:39.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:39.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:39.714 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:39.714 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:39.615+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:39.715 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:39.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T01:09:39.756 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:39.755+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:39.818 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:39.817+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:39.862 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:39.862 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:39.862 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:39.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:39.888 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:39.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:09:39.936 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:39.934+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:40.020 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:40.019+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:40.041 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:40.041 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:40.041 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:40.041 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:40.042 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:40.055 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:40.062 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:40.062 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:40.080 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:40.080 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:40.099 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:40.116 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:40.116 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:40.133 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:40.133 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:40.133 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:40.132+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:40.149 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:40.166 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:40.167 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:40.170 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:40.170 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:40.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:40.183 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:40.236 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:40.235+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:40.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:40.335 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:40.334+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:40.424 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:40.423+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:40.437 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:40.437 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:40.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:40.524 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:40.523+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:40.549 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:40.549 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:40.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:40.643 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:40.642+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:40.727 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:40.726+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:40.767 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:40.767 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:40.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:40.828 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:40.827+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:40.929 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:40.928+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:41.002 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.002 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.016 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:41.029 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:41.029+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:41.131 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:41.130+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:41.224 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.225 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.231 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:41.230+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:41.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:41.332 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:41.331+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:41.359 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.359 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.368 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:41.433 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:41.432+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:41.511 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.511 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.519 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:41.564 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:41.563+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:41.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:09:41.624 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.625 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.633 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:41.674 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:41.673+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:41.735 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:41.734+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:41.780 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.780 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:41.801 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-18T01:09:41.836 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:41.835+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:41.894 INFO:teuthology.orchestra.run.smithi120.stdout:3328 write 0x79eb13 thru 0x7ac683 (0xdb71 bytes) 2024-09-18T01:09:41.903 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:41.903 INFO:teuthology.orchestra.run.smithi120.stdout:3333 punch from 0x81d4ab to 0x820dce, (0x3923 bytes) 2024-09-18T01:09:41.903 INFO:teuthology.orchestra.run.smithi120.stdout:3334 read 0x5995305 thru 0x5999d19 (0x4a15 bytes) 2024-09-18T01:09:41.914 INFO:teuthology.orchestra.run.smithi120.stdout:3335 write 0xe0d01ee thru 0xe0d74c6 (0x72d9 bytes) 2024-09-18T01:09:41.924 INFO:teuthology.orchestra.run.smithi120.stdout:3336 read 0x8d02cee thru 0x8d093da (0x66ed bytes) 2024-09-18T01:09:41.930 INFO:teuthology.orchestra.run.smithi120.stdout:3337 trunc from 0xe0d74c7 to 0x9e91285 2024-09-18T01:09:41.937 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:41.936+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:41.939 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:41.939 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:41.943 INFO:teuthology.orchestra.run.smithi120.stdout:3342 read 0x2f80fd5 thru 0x2f8353c (0x2568 bytes) 2024-09-18T01:09:41.948 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:41.949 INFO:teuthology.orchestra.run.smithi120.stdout:3344 trunc from 0x9e91285 to 0x4b052f5 2024-09-18T01:09:41.961 INFO:teuthology.orchestra.run.smithi120.stdout:3345 read 0x37c2d61 thru 0x37d1ce7 (0xef87 bytes) 2024-09-18T01:09:41.975 INFO:teuthology.orchestra.run.smithi120.stdout:3346 writesame 0xc10e2c1 thru 0xc118051 data_size 0x13(0x9d91 bytes) 2024-09-18T01:09:41.977 INFO:teuthology.orchestra.run.smithi120.stdout:3347 trunc from 0xc118052 to 0xe236e87 2024-09-18T01:09:41.984 INFO:teuthology.orchestra.run.smithi120.stdout:3348 write 0x4bb41d thru 0x4c1d25 (0x6909 bytes) 2024-09-18T01:09:41.989 INFO:teuthology.orchestra.run.smithi120.stdout:3349 write 0x8bdecc4 thru 0x8be922e (0xa56b bytes) 2024-09-18T01:09:41.998 INFO:teuthology.orchestra.run.smithi120.stdout:3350 writesame 0x6632c2c thru 0x663e327 data_size 0x62(0xb6fc bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3352 read 0x8513419 thru 0x8518104 (0x4cec bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3353 punch from 0x75dce60 to 0x75dd064, (0x204 bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3354 punch from 0x6c8c4da to 0x6c9002e, (0x3b54 bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3355 read 0x9e3dc35 thru 0x9e48a67 (0xae33 bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3360 punch from 0x967552c to 0x9680f32, (0xba06 bytes) 2024-09-18T01:09:42.009 INFO:teuthology.orchestra.run.smithi120.stdout:3362 write 0x9866c9a thru 0x98739d9 (0xcd40 bytes) 2024-09-18T01:09:42.021 INFO:teuthology.orchestra.run.smithi120.stdout:3364 writesame 0x8be0d89 thru 0x8bee967 data_size 0x77(0xdbdf bytes) 2024-09-18T01:09:42.032 INFO:teuthology.orchestra.run.smithi120.stdout:3366 write 0x93d8d7e thru 0x93deb30 (0x5db3 bytes) 2024-09-18T01:09:42.037 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:42.037+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:42.046 INFO:teuthology.orchestra.run.smithi120.stdout:3367 read 0xd8aa5fb thru 0xd8b1602 (0x7008 bytes) 2024-09-18T01:09:42.046 INFO:teuthology.orchestra.run.smithi120.stdout:3368 read 0x9acae77 thru 0x9acd761 (0x28eb bytes) 2024-09-18T01:09:42.046 INFO:teuthology.orchestra.run.smithi120.stdout:3369 punch from 0xc8a97c0 to 0xc8aae0c, (0x164c bytes) 2024-09-18T01:09:42.047 INFO:teuthology.orchestra.run.smithi120.stdout:3371 read 0xfa6c5d thru 0xfb135e (0xa702 bytes) 2024-09-18T01:09:42.049 INFO:teuthology.orchestra.run.smithi120.stdout:3372 punch from 0x8570486 to 0x858009d, (0xfc17 bytes) 2024-09-18T01:09:42.049 INFO:teuthology.orchestra.run.smithi120.stdout:3376 write 0xb617771 thru 0xb61f36e (0x7bfe bytes) 2024-09-18T01:09:42.061 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.061 INFO:teuthology.orchestra.run.smithi120.stdout:3380 write 0x9e0ba2f thru 0x9e1afe7 (0xf5b9 bytes) 2024-09-18T01:09:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:3381 punch from 0x24bb306 to 0x24bb931, (0x62b bytes) 2024-09-18T01:09:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:3384 read 0xcd2e48b thru 0xcd30948 (0x24be bytes) 2024-09-18T01:09:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:3388 writesame 0x2b28d12 thru 0x2b33e67 data_size 0x1(0xb156 bytes) 2024-09-18T01:09:42.067 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:42.067 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:42.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:42.086 INFO:teuthology.orchestra.run.smithi120.stdout:3389 writesame 0x7e383bc thru 0x7e3dd0a data_size 0x1(0x594f bytes) 2024-09-18T01:09:42.097 INFO:teuthology.orchestra.run.smithi120.stdout:3390 writesame 0x77fffc9 thru 0x7807958 data_size 0x28(0x7990 bytes) 2024-09-18T01:09:42.107 INFO:teuthology.orchestra.run.smithi120.stdout:3391 write 0x8a989ab thru 0x8aa7543 (0xeb99 bytes) 2024-09-18T01:09:42.115 INFO:teuthology.orchestra.run.smithi120.stdout:3392 writesame 0x6310aa8 thru 0x6315760 data_size 0x1(0x4cb9 bytes) 2024-09-18T01:09:42.123 INFO:teuthology.orchestra.run.smithi120.stdout:3393 trunc from 0xe236e87 to 0xa334e72 2024-09-18T01:09:42.138 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:42.137+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:42.143 INFO:teuthology.orchestra.run.smithi120.stdout:3395 write 0xd59eb11 thru 0xd5a6062 (0x7552 bytes) 2024-09-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi120.stdout:3396 trunc from 0xd5a6063 to 0x87ab853 2024-09-18T01:09:42.174 INFO:teuthology.orchestra.run.smithi120.stdout:3397 write 0x269223a thru 0x269f23b (0xd002 bytes) 2024-09-18T01:09:42.183 INFO:teuthology.orchestra.run.smithi120.stdout:3398 read 0x5a54816 thru 0x5a5b9e3 (0x71ce bytes) 2024-09-18T01:09:42.183 INFO:teuthology.orchestra.run.smithi120.stdout:3399 write 0x3b74c2b thru 0x3b79108 (0x44de bytes) 2024-09-18T01:09:42.186 INFO:teuthology.orchestra.run.smithi120.stdout:3400 read 0x522ae18 thru 0x522d40a (0x25f3 bytes) 2024-09-18T01:09:42.186 INFO:teuthology.orchestra.run.smithi120.stdout:3401 punch from 0x3a813c6 to 0x3a891a6, (0x7de0 bytes) 2024-09-18T01:09:42.186 INFO:teuthology.orchestra.run.smithi120.stdout:3402 punch from 0x17367bf to 0x173aeff, (0x4740 bytes) 2024-09-18T01:09:42.194 INFO:teuthology.orchestra.run.smithi120.stdout:3403 trunc from 0x87ab853 to 0xe674a4 2024-09-18T01:09:42.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:42.198 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:42.198 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:42.209 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:42.239 INFO:teuthology.orchestra.run.smithi120.stdout:3404 writesame 0x37d3432 thru 0x37d9293 data_size 0x3(0x5e62 bytes) 2024-09-18T01:09:42.240 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:42.239+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:42.246 INFO:teuthology.orchestra.run.smithi120.stdout:3405 write 0x45285c9 thru 0x452a2d5 (0x1d0d bytes) 2024-09-18T01:09:42.258 INFO:teuthology.orchestra.run.smithi120.stdout:3407 trunc from 0x452a2d6 to 0x247f5dd 2024-09-18T01:09:42.294 INFO:teuthology.orchestra.run.smithi120.stdout:3409 write 0xdacb3aa thru 0xdad60f9 (0xad50 bytes) 2024-09-18T01:09:42.306 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.306 INFO:teuthology.orchestra.run.smithi120.stdout:3412 read 0xa22f19c thru 0xa22f2e0 (0x145 bytes) 2024-09-18T01:09:42.313 INFO:teuthology.orchestra.run.smithi120.stdout:3413 writesame 0xe3fb64b thru 0xe4098d0 data_size 0xf(0xe286 bytes) 2024-09-18T01:09:42.313 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.313 INFO:teuthology.orchestra.run.smithi120.stdout:3415 read 0xb979b87 thru 0xb97c496 (0x2910 bytes) 2024-09-18T01:09:42.313 INFO:teuthology.orchestra.run.smithi120.stdout:3416 write 0x80b44f thru 0x80c8ed (0x149f bytes) 2024-09-18T01:09:42.315 INFO:teuthology.orchestra.run.smithi120.stdout:3417 read 0x5c536d8 thru 0x5c62681 (0xefaa bytes) 2024-09-18T01:09:42.315 INFO:teuthology.orchestra.run.smithi120.stdout:3419 punch from 0x6ca5182 to 0x6ca80f1, (0x2f6f bytes) 2024-09-18T01:09:42.315 INFO:teuthology.orchestra.run.smithi120.stdout:3420 read 0x8eee7ef thru 0x8ef3de5 (0x55f7 bytes) 2024-09-18T01:09:42.315 INFO:teuthology.orchestra.run.smithi120.stdout:3421 trunc from 0xe4098d1 to 0xdecd15c 2024-09-18T01:09:42.328 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.328 INFO:teuthology.orchestra.run.smithi120.stdout:3424 read 0x436e0b7 thru 0x436ead8 (0xa22 bytes) 2024-09-18T01:09:42.328 INFO:teuthology.orchestra.run.smithi120.stdout:3425 writesame 0x1e154cf thru 0x1e1f0dc data_size 0x55(0x9c0e bytes) 2024-09-18T01:09:42.337 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.338 INFO:teuthology.orchestra.run.smithi120.stdout:3429 write 0x7056065 thru 0x7056af2 (0xa8e bytes) 2024-09-18T01:09:42.339 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:42.339+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:42.341 INFO:teuthology.orchestra.run.smithi120.stdout:3430 read 0xbd26d87 thru 0xbd2e479 (0x76f3 bytes) 2024-09-18T01:09:42.341 INFO:teuthology.orchestra.run.smithi120.stdout:3431 write 0x5482856 thru 0x5483dff (0x15aa bytes) 2024-09-18T01:09:42.346 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:42.346 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:42.347 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.347 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:42.347 INFO:teuthology.orchestra.run.smithi120.stdout:3435 write 0xc5c46ea thru 0xc5d41da (0xfaf1 bytes) 2024-09-18T01:09:42.358 INFO:teuthology.orchestra.run.smithi120.stdout:3436 write 0x5c96ca9 thru 0x5c98845 (0x1b9d bytes) 2024-09-18T01:09:42.362 INFO:teuthology.orchestra.run.smithi120.stdout:3437 writesame 0x2f60c7d thru 0x2f64242 data_size 0x1(0x35c6 bytes) 2024-09-18T01:09:42.365 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:42.365 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:42.368 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:3438 read 0xb2ed5dc thru 0xb2f53c7 (0x7dec bytes) 2024-09-18T01:09:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:3440 write 0x66ad4ef thru 0x66b47cb (0x72dd bytes) 2024-09-18T01:09:42.376 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:42.380 INFO:teuthology.orchestra.run.smithi120.stdout:3441 punch from 0x34216df to 0x342755c, (0x5e7d bytes) 2024-09-18T01:09:42.381 INFO:teuthology.orchestra.run.smithi120.stdout:3442 write 0xcef370f thru 0xcefad25 (0x7617 bytes) 2024-09-18T01:09:42.390 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:42.391 INFO:teuthology.orchestra.run.smithi120.stdout:3443 write 0xaa0d73 thru 0xaaa9d3 (0x9c61 bytes) 2024-09-18T01:09:42.394 INFO:teuthology.orchestra.run.smithi120.stdout:3444 write 0x916483b thru 0x917419b (0xf961 bytes) 2024-09-18T01:09:42.404 INFO:teuthology.orchestra.run.smithi120.stdout:3445 writesame 0xa2c240a thru 0xa2cecf0 data_size 0x1(0xc8e7 bytes) 2024-09-18T01:09:42.412 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:42.417 INFO:teuthology.orchestra.run.smithi120.stdout:3446 writesame 0x3f4020c thru 0x3f43435 data_size 0x1(0x322a bytes) 2024-09-18T01:09:42.431 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:42.432 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:42.432 INFO:teuthology.orchestra.run.smithi120.stdout:3447 punch from 0x9142dee to 0x914e47e, (0xb690 bytes) 2024-09-18T01:09:42.432 INFO:teuthology.orchestra.run.smithi120.stdout:3448 read 0x1456d8c thru 0x145eec6 (0x813b bytes) 2024-09-18T01:09:42.432 INFO:teuthology.orchestra.run.smithi120.stdout:3451 write 0x33bb85e thru 0x33c34a1 (0x7c44 bytes) 2024-09-18T01:09:42.440 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:42.439+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:42.444 INFO:teuthology.orchestra.run.smithi120.stdout:3452 clone 32 order 20 su 131072 sc 12 2024-09-18T01:09:42.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:42.452 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:42.468 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:42.468 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:42.484 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:42.491 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.918448827000006, 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.866457683999997, 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.866380887999995, 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.861875734999998, 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:42.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.861619938999993, 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:42.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:42.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.857594610999996, 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:42.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.857453461000006, 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:42.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.857220557000005, 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:42.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:42.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.854601494999997, 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:42.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:42.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.832262460999999, 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:42.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881747187000002, 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:42.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881725445999997, 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:42.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:42.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:42.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881705038, 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:42.552 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881685378999997, 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:42.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:42.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:42.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881652043999999, 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:42.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:42.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881617511999998, 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:42.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881572622, 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:42.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:42.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881546247000003, 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:42.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:42.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:42.579 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881527095999999, 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.881508527999998, 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:42.584 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:42.588 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:42.590 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:42.589+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:42.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:42.691 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:42.690+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:09:42.737 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:42.738 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:42.747 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:42.831 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:42.830+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:42.891 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:42.891 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:42.891 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:42.891 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:42.891 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:42.892 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:42.892+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:42.899 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:42.993 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:42.992+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 151.144324942, 2024-09-18T01:09:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:09:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:09:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:09:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:09:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65806 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 147456~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420808108999999, 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054191588999999998, 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65806 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621326+0000", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621328+0000", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0770000000000002e-06 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621335+0000", 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.221e-06 2024-09-18T01:09:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621341+0000", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3070000000000002e-06 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675290+0000", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053949583000000002 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675341+0000", 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0933999999999997e-05 2024-09-18T01:09:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675517+0000", 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017606100000000001 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65807 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 169984~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420786698000001, 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054422048000000001, 2024-09-18T01:09:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65807 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621347+0000", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621349+0000", 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621356+0000", 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621362+0000", 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675543+0000", 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054181172 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675589+0000", 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6208000000000002e-05 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675769+0000", 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018009699999999999 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65808 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 192512~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420763782999998, 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054655414999999999, 2024-09-18T01:09:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65808 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621370+0000", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621373+0000", 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2450000000000001e-06 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621379+0000", 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621383+0000", 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675795+0000", 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054411302000000002 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.675846+0000", 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1180999999999997e-05 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676025+0000", 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179424 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65809 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 215040~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420742765999996, 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054910481999999997, 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65809 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621391+0000", 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:09:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621393+0000", 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.641e-06 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621401+0000", 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6310000000000003e-06 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621406+0000", 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0270000000000003e-06 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676057+0000", 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054651648999999997 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676110+0000", 2024-09-18T01:09:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2945000000000002e-05 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676301+0000", 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000191182 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65810 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 237568~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420711558999997, 2024-09-18T01:09:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055159933000000001, 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65810 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621422+0000", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621423+0000", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621425+0000", 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621433+0000", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621437+0000", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.809e-06 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676331+0000", 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.054893465000000002 2024-09-18T01:09:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676377+0000", 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5921999999999998e-05 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676582+0000", 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020518500000000001 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65811 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 260096~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420689230999997, 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055415161999999997, 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65811 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621444+0000", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621445+0000", 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621447+0000", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2239999999999998e-06 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621455+0000", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621460+0000", 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-06 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676614+0000", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055154659000000002 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676666+0000", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1489000000000001e-05 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676860+0000", 2024-09-18T01:09:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019366899999999999 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65812 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 282624~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420667207000001, 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055664620999999997, 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65812 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621466+0000", 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621467+0000", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621475+0000", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8420000000000005e-06 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621483+0000", 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1750000000000005e-06 2024-09-18T01:09:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621488+0000", 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1309999999999997e-06 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676903+0000", 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055414897999999997 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.676945+0000", 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2286000000000001e-05 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677131+0000", 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018574999999999999 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65813 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 305152~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420617780000001, 2024-09-18T01:09:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055863346000000001, 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65813 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621516+0000", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621520+0000", 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2370000000000001e-06 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621530+0000", 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0287e-05 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621536+0000", 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677161+0000", 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055624280999999998 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677204+0000", 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3704e-05 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677379+0000", 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174897 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65814 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 327680~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420586683000003, 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056100530000000003, 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65814 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621547+0000", 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621550+0000", 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621558+0000", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9249999999999995e-06 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621563+0000", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677408+0000", 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.055844700999999997 2024-09-18T01:09:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677452+0000", 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3404e-05 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677647+0000", 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195973 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65815 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 350208~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420563688000001, 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056216462000000002, 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65815 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621570+0000", 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621572+0000", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621580+0000", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4830000000000002e-06 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621585+0000", 2024-09-18T01:09:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2190000000000003e-06 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677677+0000", 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056092026000000003 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677721+0000", 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4208000000000001e-05 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677786+0000", 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5030000000000006e-05 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65816 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 372736~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420541970000002, 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056532801000000001, 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65816 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621592+0000", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621594+0000", 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621602+0000", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6669999999999996e-06 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621607+0000", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1379999999999998e-06 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677810+0000", 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056203257 2024-09-18T01:09:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.677895+0000", 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.462e-05 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678124+0000", 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000229633 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65817 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 395264~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420520549000003, 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056779646000000003, 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65817 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621613+0000", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621614+0000", 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621616+0000", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621623+0000", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4580000000000004e-06 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621628+0000", 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:09:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678153+0000", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056525059000000002 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678198+0000", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4218000000000003e-05 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678393+0000", 2024-09-18T01:09:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195212 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65818 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 417792~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420499392000004, 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056893621999999998, 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65818 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621634+0000", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621635+0000", 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621637+0000", 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621647+0000", 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0565e-05 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621653+0000", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678422+0000", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056769009000000002 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678464+0000", 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2796999999999997e-05 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678528+0000", 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3403000000000003e-05 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65819 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 440320~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420474413000001, 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057086459999999999, 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.109 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65819 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621659+0000", 2024-09-18T01:09:43.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621660+0000", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621662+0000", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2419999999999999e-06 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621678+0000", 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5756999999999998e-05 2024-09-18T01:09:43.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621683+0000", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678564+0000", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.056880848999999997 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678644+0000", 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0273999999999997e-05 2024-09-18T01:09:43.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678746+0000", 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101399 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65820 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 462848~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420443089999999, 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057188359000000001, 2024-09-18T01:09:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65820 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621691+0000", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621694+0000", 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4679999999999999e-06 2024-09-18T01:09:43.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621702+0000", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2519999999999999e-06 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621707+0000", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1370000000000001e-06 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678775+0000", 2024-09-18T01:09:43.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057067937999999999 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678819+0000", 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3665999999999998e-05 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678879+0000", 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-05 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65821 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 485376~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420420200999999, 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057383643999999998, 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65821 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621713+0000", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621714+0000", 2024-09-18T01:09:43.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621716+0000", 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0320000000000002e-06 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621724+0000", 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8849999999999999e-06 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621729+0000", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4480000000000002e-06 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678919+0000", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057189443999999999 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.678998+0000", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8998000000000005e-05 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679097+0000", 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9446999999999998e-05 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420396572000001, 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:43.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.420374391000003, 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:09:43.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:09:43.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.452869924, 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:43.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:43.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:09:43.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:43.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:09:43.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:43.137 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:43.139 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:43.139 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:43.138+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:43.240 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:43.239+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:43.243 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:43.243 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:43.243 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:43.244 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:43.252 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:43.341 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:43.340+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:43.354 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:43.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:43.354 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:43.354 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:43.354 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:43.362 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:43.442 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:43.441+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:43.466 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:8372 3.1 3:843bd5bb:::rbd_data.11762d3aac1.0000000000000003:head [write 0~2097152 [fadvise_sequential+fadvise_nocache],zero 2097152~524288,write 2621440~246948 [fadvise_sequential+fadvise_nocache],zero 2868388~801628,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~589824,write 6356992~32768 [fadvise_sequential+fadvise_nocache],zero 6389760~98304,write 6488064~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 122.868862537, 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.041091346000000001, 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 8372 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596228+0000", 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.596229+0000", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597365+0000", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011360739999999999 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597385+0000", 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9471e-05 2024-09-18T01:09:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.597397+0000", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2544000000000001e-05 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600026+0000", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026284519999999999 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600199+0000", 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017387500000000001 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.600763+0000", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056360799999999995 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.610856+0000", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010092985 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637159+0000", 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.026303 2024-09-18T01:09:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637235+0000", 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5576999999999998e-05 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.637320+0000", 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5226e-05 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:9168 3.1f 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~3670016 [fadvise_sequential+fadvise_nocache],zero 5242880~81920,write 5324800~32768 [fadvise_sequential+fadvise_nocache],zero 5357568~98304,write 5455872~32768 [fadvise_sequential+fadvise_nocache],zero 5505024~81920,write 5586944~32768 [fadvise_sequential+fadvise_nocache],zero 5619712~98304,write 5718016~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 122.807259915, 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.033614088, 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 9168 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657831+0000", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.657833+0000", 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2520000000000002e-06 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660837+0000", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030036210000000002 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660874+0000", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7463999999999999e-05 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660890+0000", 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5124e-05 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.660986+0000", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6324999999999997e-05 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.661626+0000", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00064029099999999997 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.662014+0000", 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038804900000000001 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.685075+0000", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023060536 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691368+0000", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062932439999999999 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691415+0000", 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7042000000000001e-05 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:40.691445+0000", 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0139999999999999e-05 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.742410527, 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72452 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1875968~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500935888999997, 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029616995, 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72452 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964155+0000", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964156+0000", 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964158+0000", 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964165+0000", 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4420000000000001e-06 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964171+0000", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0020000000000003e-06 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993555+0000", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029384161999999998 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993644+0000", 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8378e-05 2024-09-18T01:09:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993772+0000", 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128458 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72453 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1898496~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500911723999998, 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029836705000000002, 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72453 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964179+0000", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964180+0000", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.51e-07 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964182+0000", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-06 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964189+0000", 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2830000000000004e-06 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964195+0000", 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2379999999999997e-06 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993800+0000", 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029605474999999999 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.993873+0000", 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2679e-05 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994016+0000", 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014327200000000001 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72454 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1921024~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500890616, 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063757, 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72454 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964200+0000", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964201+0000", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9999999999999998e-07 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964203+0000", 2024-09-18T01:09:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1119999999999999e-06 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964210+0000", 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2119999999999997e-06 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964215+0000", 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0409999999999997e-06 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994046+0000", 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029830837999999998 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994145+0000", 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9130999999999997e-05 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994264+0000", 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119023 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72455 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1943552~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500869743000003, 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030250270999999999, 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72455 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964221+0000", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964222+0000", 2024-09-18T01:09:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964224+0000", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.136e-06 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964229+0000", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964233+0000", 2024-09-18T01:09:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7809999999999996e-06 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994297+0000", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030063145999999999 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994373+0000", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6561000000000001e-05 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994472+0000", 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8364999999999998e-05 2024-09-18T01:09:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72456 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1966080~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500851517000001, 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030387272999999999, 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72456 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964239+0000", 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964240+0000", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964242+0000", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964249+0000", 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0310000000000001e-06 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964255+0000", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994502+0000", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030246538 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994569+0000", 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6673000000000004e-05 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994627+0000", 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8106999999999998e-05 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72457 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1988608~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500759346999999, 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030506231000000002, 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72457 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964332+0000", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964336+0000", 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3670000000000001e-06 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964345+0000", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1220000000000001e-06 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964351+0000", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994657+0000", 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030306777999999999 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994747+0000", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9370999999999995e-05 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994838+0000", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72458 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2011136~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500730935999997, 2024-09-18T01:09:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030618253000000002, 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72458 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964360+0000", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964363+0000", 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964370+0000", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964375+0000", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3770000000000004e-06 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994859+0000", 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030483999000000001 2024-09-18T01:09:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994901+0000", 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1726e-05 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994978+0000", 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7348999999999994e-05 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72459 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2033664~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500708945, 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030730760999999999, 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72459 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964382+0000", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964385+0000", 2024-09-18T01:09:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964391+0000", 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964396+0000", 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9169999999999997e-06 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.994997+0000", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030601225999999999 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995034+0000", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7055999999999999e-05 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995113+0000", 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8572999999999995e-05 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72460 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2056192~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500688963999998, 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030845698000000001, 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72460 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964402+0000", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964403+0000", 2024-09-18T01:09:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964405+0000", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964411+0000", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964416+0000", 2024-09-18T01:09:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5029999999999997e-06 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995132+0000", 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030715527999999999 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995173+0000", 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0601e-05 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995248+0000", 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5179000000000006e-05 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72461 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2078720~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.500668107000003, 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030963408000000001, 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72461 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964423+0000", 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964426+0000", 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2859999999999998e-06 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964432+0000", 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.964437+0000", 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995266+0000", 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030829143999999999 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995309+0000", 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3214000000000003e-05 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.995386+0000", 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7361999999999996e-05 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72590 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1968128~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495892466999997, 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029714720999999999, 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72590 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969199+0000", 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969201+0000", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969212+0000", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1231e-05 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969218+0000", 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998815+0000", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029597073000000002 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998856+0000", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0911000000000002e-05 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998913+0000", 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7747999999999997e-05 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72591 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 1990656~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495828650999997, 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.02984761, 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72591 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969262+0000", 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969263+0000", 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.68e-07 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969266+0000", 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969274+0000", 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0749999999999998e-06 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969279+0000", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.626e-06 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.998946+0000", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029666446999999999 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999049+0000", 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010286 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999110+0000", 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72592 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2013184~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495803406999997, 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029985181, 2024-09-18T01:09:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72592 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969288+0000", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969290+0000", 2024-09-18T01:09:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9889999999999999e-06 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969297+0000", 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0859999999999996e-06 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969316+0000", 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.914e-05 2024-09-18T01:09:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999141+0000", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029825122999999999 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999212+0000", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0840000000000006e-05 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999273+0000", 2024-09-18T01:09:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0575000000000002e-05 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72593 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2035712~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495764659999999, 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030079117999999998, 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72593 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969326+0000", 2024-09-18T01:09:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969327+0000", 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.96e-07 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969329+0000", 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3949999999999999e-06 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969336+0000", 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0339999999999999e-06 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969341+0000", 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2619999999999998e-06 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999304+0000", 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029962231999999998 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999348+0000", 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4703999999999998e-05 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999405+0000", 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7095000000000001e-05 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72594 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2058240~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495742344999996, 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030207385999999999, 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72594 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969349+0000", 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969351+0000", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969358+0000", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969363+0000", 2024-09-18T01:09:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2120000000000002e-06 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999436+0000", 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030072636 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999495+0000", 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9568000000000003e-05 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999556+0000", 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0646000000000002e-05 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72595 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:34 [read 2080768~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495719835000003, 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030335257000000001, 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72595 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969371+0000", 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969372+0000", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.03e-07 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969387+0000", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5162e-05 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969394+0000", 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6459999999999993e-06 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969400+0000", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999587+0000", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030186761999999999 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999634+0000", 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7129999999999998e-05 2024-09-18T01:09:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999706+0000", 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699000000000003e-05 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:72596 3.1 3:8776866f:::rbd_data.117658cbc921.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.495683526000001, 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030489353, 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 72596 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969407+0000", 2024-09-18T01:09:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969408+0000", 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969410+0000", 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1730000000000002e-06 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969417+0000", 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.969422+0000", 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999743+0000", 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030320732 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999813+0000", 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9831000000000003e-05 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.999897+0000", 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4289000000000006e-05 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:43.558 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:43.560 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:43.560 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:43.559+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:43.660 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:43.660+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:43.703 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:43.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:43.703 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:43.703 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:43.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:43.797 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:43.796+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:43.863 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:43.862+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:43.919 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:43.919 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:43.919 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:43.919 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:43.919 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:43.933 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:43.963 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:43.962+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:44.064 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:44.063+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:44.146 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.965436012, 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.956654063, 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367129132999999, 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.160 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:44.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.164 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367106831000001, 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:44.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:44.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:44.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367085877000001, 2024-09-18T01:09:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:44.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:44.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:44.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367065033999999, 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.174 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:44.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.178 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367043631999998, 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:44.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:44.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367024452000003, 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:44.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.367005097000003, 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.188 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:44.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.192 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366966493, 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:44.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366936354000003, 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.197 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:44.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366912900999999, 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:44.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366890884999997, 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366869917000002, 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366849059000003, 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.215 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:44.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366820609999998, 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366798469000003, 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:44.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:44.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:44.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366775113999999, 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:44.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.366752179999999, 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:44.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.363713181999998, 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:44.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.242 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:44.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:44.244 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:44.242+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:44.345 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:44.344+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:44.387 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:44.387 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:44.387 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:44.387 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:44.395 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:44.480 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:44.479+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:44.539 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:44.539 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:44.546 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:44.545+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:44.547 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:44.591 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:44.674 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:44.673+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.031955975000001, 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.695 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.031901749999999, 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:09:44.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.704 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.031869292, 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:44.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:09:44.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.031738029, 2024-09-18T01:09:44.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:09:44.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.031268043000001, 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:09:44.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:09:44.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.030105443, 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:09:44.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:09:44.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.029998161, 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:09:44.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.728 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.029916502000001, 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:09:44.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.029401114000001, 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:44.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.028581876000001, 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:44.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:44.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.4553167900000004, 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:09:44.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:09:44.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:09:44.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158932 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2977274460000001, 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073292979999999997, 2024-09-18T01:09:44.750 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158932 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395043+0000", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-07 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395045+0000", 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2019999999999998e-06 2024-09-18T01:09:44.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395048+0000", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.095e-06 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395051+0000", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.661e-06 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400622+0000", 2024-09-18T01:09:44.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055717459999999998 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400709+0000", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6984999999999995e-05 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401039+0000", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032971600000000003 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401665+0000", 2024-09-18T01:09:44.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062626499999999998 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402286+0000", 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062092099999999997 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402337+0000", 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099e-05 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402372+0000", 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.434e-05 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2884578979999999, 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.756 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:09:44.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:09:44.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2606125100000001, 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:09:44.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:44.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2605922110000001, 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:09:44.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:09:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:09:44.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2605719479999999, 2024-09-18T01:09:44.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:09:44.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.260552095, 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.779 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:09:44.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.2605329379999999, 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.785 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:09:44.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:09:44.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.876542512, 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:09:44.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:09:44.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.81658364900000002, 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:44.797 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:44.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:09:44.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:44.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:44.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:44.803 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:44.803 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:09:44.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:44.805 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:44.806 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:44.806 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:44.807 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:44.808 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:44.804+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:44.821 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:44.839 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:44.840 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:44.858 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:44.858 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:44.878 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:44.884 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-18T01:09:44.931 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:44.930+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:44.955 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:44.955 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:44.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:45.006 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:45.006+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:45.079 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.079 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.079 INFO:teuthology.orchestra.run.smithi120.stdout:3455 read 0xb0835f6 thru 0xb08d0c8 (0x9ad3 bytes) 2024-09-18T01:09:45.081 INFO:teuthology.orchestra.run.smithi120.stdout:3456 punch from 0xd1d41eb to 0xd1e2e25, (0xec3a bytes) 2024-09-18T01:09:45.084 INFO:teuthology.orchestra.run.smithi120.stdout:3457 punch from 0xcfbd863 to 0xcfc86b0, (0xae4d bytes) 2024-09-18T01:09:45.084 INFO:teuthology.orchestra.run.smithi120.stdout:3458 writesame 0x318217a thru 0x318d471 data_size 0x2e(0xb2f8 bytes) 2024-09-18T01:09:45.103 INFO:teuthology.orchestra.run.smithi120.stdout:3460 trunc from 0xdecd15c to 0x90243d 2024-09-18T01:09:45.107 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:45.106+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:45.117 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:45.117 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:45.117 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-18T01:09:45.117 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:09:45.117 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:09:45.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-18T01:09:45.139 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.139 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:45.190 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.190 INFO:teuthology.orchestra.run.smithi120.stdout:3464 punch from 0x219187 to 0x219772, (0x5eb bytes) 2024-09-18T01:09:45.190 INFO:teuthology.orchestra.run.smithi120.stdout:3466 read 0x192dfc thru 0x19dc15 (0xae1a bytes) 2024-09-18T01:09:45.191 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.191 INFO:teuthology.orchestra.run.smithi120.stdout:3471 punch from 0x67b812 to 0x6849bd, (0x91ab bytes) 2024-09-18T01:09:45.209 INFO:teuthology.orchestra.run.smithi120.stdout:3473 write 0xb660fac thru 0xb66bd5c (0xadb1 bytes) 2024-09-18T01:09:45.210 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:45.209+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:45.213 INFO:teuthology.orchestra.run.smithi120.stdout:3474 read 0xcd082d thru 0xcd3d44 (0x3518 bytes) 2024-09-18T01:09:45.213 INFO:teuthology.orchestra.run.smithi120.stdout:3475 write 0x203a34d thru 0x2045638 (0xb2ec bytes) 2024-09-18T01:09:45.220 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.220 INFO:teuthology.orchestra.run.smithi120.stdout:3477 write 0x373c2a0 thru 0x37418bf (0x5620 bytes) 2024-09-18T01:09:45.225 INFO:teuthology.orchestra.run.smithi120.stdout:3479 write 0x174de4 thru 0x17e1be (0x93db bytes) 2024-09-18T01:09:45.227 INFO:teuthology.orchestra.run.smithi120.stdout:3480 punch from 0x3225d20 to 0x322becc, (0x61ac bytes) 2024-09-18T01:09:45.228 INFO:teuthology.orchestra.run.smithi120.stdout:3482 trunc from 0xb66bd5d to 0xa54659d 2024-09-18T01:09:45.239 INFO:teuthology.orchestra.run.smithi120.stdout:3483 write 0x52b1c67 thru 0x52bbd7d (0xa117 bytes) 2024-09-18T01:09:45.243 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.243 INFO:teuthology.orchestra.run.smithi120.stdout:3490 read 0x13c18d0 thru 0x13d0073 (0xe7a4 bytes) 2024-09-18T01:09:45.243 INFO:teuthology.orchestra.run.smithi120.stdout:3492 read 0x53c0bc4 thru 0x53ccba8 (0xbfe5 bytes) 2024-09-18T01:09:45.243 INFO:teuthology.orchestra.run.smithi120.stdout:3493 writesame 0x70eb2a5 thru 0x70f4464 data_size 0x58(0x91c0 bytes) 2024-09-18T01:09:45.248 INFO:teuthology.orchestra.run.smithi120.stdout:3494 read 0x19d30ba thru 0x19e0a41 (0xd988 bytes) 2024-09-18T01:09:45.249 INFO:teuthology.orchestra.run.smithi120.stdout:3495 read 0x9d3557b thru 0x9d43157 (0xdbdd bytes) 2024-09-18T01:09:45.249 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.249 INFO:teuthology.orchestra.run.smithi120.stdout:3498 write 0x6331965 thru 0x6331ac4 (0x160 bytes) 2024-09-18T01:09:45.250 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:45.250 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:09:45.251 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:45.252 INFO:teuthology.orchestra.run.smithi120.stdout:3499 read 0x2e10538 thru 0x2e1a4eb (0x9fb4 bytes) 2024-09-18T01:09:45.252 INFO:teuthology.orchestra.run.smithi120.stdout:3500 write 0xc9336f thru 0xc95c45 (0x28d7 bytes) 2024-09-18T01:09:45.255 INFO:teuthology.orchestra.run.smithi120.stdout:3502 punch from 0x51b1dcf to 0x51ba6fa, (0x892b bytes) 2024-09-18T01:09:45.255 INFO:teuthology.orchestra.run.smithi120.stdout:3503 writesame 0x9fbe5e0 thru 0x9fcb9a8 data_size 0x1(0xd3c9 bytes) 2024-09-18T01:09:45.281 INFO:teuthology.orchestra.run.smithi120.stdout:3504 punch from 0x3ee2bf1 to 0x3ee3f35, (0x1344 bytes) 2024-09-18T01:09:45.281 INFO:teuthology.orchestra.run.smithi120.stdout:3505 punch from 0x8e7e85e to 0x8e87060, (0x8802 bytes) 2024-09-18T01:09:45.282 INFO:teuthology.orchestra.run.smithi120.stdout:3506 writesame 0x2562831 thru 0x2570336 data_size 0x7e(0xdb06 bytes) 2024-09-18T01:09:45.285 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.295 INFO:teuthology.orchestra.run.smithi120.stdout:3508 write 0xe208ba2 thru 0xe217acf (0xef2e bytes) 2024-09-18T01:09:45.299 INFO:teuthology.orchestra.run.smithi120.stdout:3509 punch from 0x14549b8 to 0x1455155, (0x79d bytes) 2024-09-18T01:09:45.299 INFO:teuthology.orchestra.run.smithi120.stdout:3513 read 0x45c0e9 thru 0x46081e (0x4736 bytes) 2024-09-18T01:09:45.299 INFO:teuthology.orchestra.run.smithi120.stdout:3514 read 0x767dc01 thru 0x7689273 (0xb673 bytes) 2024-09-18T01:09:45.299 INFO:teuthology.orchestra.run.smithi120.stdout:3516 punch from 0x84ee3c9 to 0x84fe373, (0xffaa bytes) 2024-09-18T01:09:45.299 INFO:teuthology.orchestra.run.smithi120.stdout:3517 punch from 0xa233a18 to 0xa23ca62, (0x904a bytes) 2024-09-18T01:09:45.300 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.308 INFO:teuthology.orchestra.run.smithi120.stdout:3519 trunc from 0xe217ad0 to 0x72e7bda 2024-09-18T01:09:45.308 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:45.307+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:45.311 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.311 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.320 INFO:teuthology.orchestra.run.smithi120.stdout:3521 read 0x682725e thru 0x682d09f (0x5e42 bytes) 2024-09-18T01:09:45.321 INFO:teuthology.orchestra.run.smithi120.stdout:3522 read 0x1c4b456 thru 0x1c5a09c (0xec47 bytes) 2024-09-18T01:09:45.322 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:45.322 INFO:teuthology.orchestra.run.smithi120.stdout:3523 read 0xd68df thru 0xdd372 (0x6a94 bytes) 2024-09-18T01:09:45.322 INFO:teuthology.orchestra.run.smithi120.stdout:3524 read 0x1d4395b thru 0x1d4afda (0x7680 bytes) 2024-09-18T01:09:45.333 INFO:teuthology.orchestra.run.smithi120.stdout:3526 writesame 0xc67d681 thru 0xc686aa1 data_size 0x1(0x9421 bytes) 2024-09-18T01:09:45.340 INFO:teuthology.orchestra.run.smithi120.stdout:3528 trunc from 0xc686aa2 to 0x6ee72b4 2024-09-18T01:09:45.353 INFO:teuthology.orchestra.run.smithi120.stdout:3529 trunc from 0x6ee72b4 to 0xb7921cc 2024-09-18T01:09:45.358 INFO:teuthology.orchestra.run.smithi120.stdout:3530 write 0x9ec5214 thru 0x9ed0d6c (0xbb59 bytes) 2024-09-18T01:09:45.362 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.362 INFO:teuthology.orchestra.run.smithi120.stdout:3532 write 0x38f4cc thru 0x39c17b (0xccb0 bytes) 2024-09-18T01:09:45.366 INFO:teuthology.orchestra.run.smithi120.stdout:3535 read 0x9c618bf thru 0x9c64308 (0x2a4a bytes) 2024-09-18T01:09:45.366 INFO:teuthology.orchestra.run.smithi120.stdout:3536 read 0x986283a thru 0x9872749 (0xff10 bytes) 2024-09-18T01:09:45.366 INFO:teuthology.orchestra.run.smithi120.stdout:3537 read 0xc17149 thru 0xc1ff54 (0x8e0c bytes) 2024-09-18T01:09:45.367 INFO:teuthology.orchestra.run.smithi120.stdout:3538 write 0x1f560b2 thru 0x1f63e0d (0xdd5c bytes) 2024-09-18T01:09:45.370 INFO:teuthology.orchestra.run.smithi120.stdout:3540 punch from 0x7f95452 to 0x7f96a15, (0x15c3 bytes) 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:3541 read 0x388da99 thru 0x388eba9 (0x1111 bytes) 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:3542 read 0x2373d8b thru 0x237d178 (0x93ee bytes) 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:3544 read 0x9a13a06 thru 0x9a23608 (0xfc03 bytes) 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:3545 read 0x92b3e4c thru 0x92be69b (0xa850 bytes) 2024-09-18T01:09:45.371 INFO:teuthology.orchestra.run.smithi120.stdout:3547 writesame 0x79f6929 thru 0x79f9580 data_size 0x18(0x2c58 bytes) 2024-09-18T01:09:45.373 INFO:teuthology.orchestra.run.smithi120.stdout:3550 read 0x9eb8ec8 thru 0x9ec7ad5 (0xec0e bytes) 2024-09-18T01:09:45.374 INFO:teuthology.orchestra.run.smithi120.stdout:3552 read 0x8de68f6 thru 0x8df6447 (0xfb52 bytes) 2024-09-18T01:09:45.374 INFO:teuthology.orchestra.run.smithi120.stdout:3553 write 0x243f7eb thru 0x244347d (0x3c93 bytes) 2024-09-18T01:09:45.380 INFO:teuthology.orchestra.run.smithi120.stdout:3554 trunc from 0xb7921cc to 0xa01ca6f 2024-09-18T01:09:45.389 INFO:teuthology.orchestra.run.smithi120.stdout:3555 write 0x47c29cb thru 0x47c7f8d (0x55c3 bytes) 2024-09-18T01:09:45.403 INFO:teuthology.orchestra.run.smithi120.stdout:3556 write 0xe1fa2d7 thru 0xe20496b (0xa695 bytes) 2024-09-18T01:09:45.407 INFO:teuthology.orchestra.run.smithi120.stdout:3559 read 0xdf1e0bd thru 0xdf2cc8e (0xebd2 bytes) 2024-09-18T01:09:45.408 INFO:teuthology.orchestra.run.smithi120.stdout:3560 write 0xd1160a0 thru 0xd1207ac (0xa70d bytes) 2024-09-18T01:09:45.408 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:45.407+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:45.412 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.562 INFO:teuthology.orchestra.run.smithi120.stdout:3562 write 0x163a76c thru 0x1640104 (0x5999 bytes) 2024-09-18T01:09:45.565 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.565 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.565 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:45.566 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:45.563+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:3563 trunc from 0xe20496c to 0x84cd6aa 2024-09-18T01:09:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:3565 trunc from 0x84cd6aa to 0xaf38d12 2024-09-18T01:09:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:3566 read 0xa34d663 thru 0xa3531c6 (0x5b64 bytes) 2024-09-18T01:09:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:3568 writesame 0x663048a thru 0x663b1c7 data_size 0x19(0xad3e bytes) 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:3569 writesame 0x3b0e176 thru 0x3b14bde data_size 0x1(0x6a69 bytes) 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:3570 read 0x5a4b96e thru 0x5a4e4e2 (0x2b75 bytes) 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:3572 writesame 0xa4944e4 thru 0xa49b03b data_size 0x28(0x6b58 bytes) 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:3573 write 0x5d17022 thru 0x5d220ab (0xb08a bytes) 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:45.567 INFO:teuthology.orchestra.run.smithi120.stdout:3575 clone 33 order 22 su 1048576 sc 5 2024-09-18T01:09:45.665 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:45.664+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:45.708 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.709 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.717 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:45.724 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:09:45.779 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:45.778+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:45.860 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.860 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.866 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:45.866+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:45.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:45.968 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:45.967+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:45.971 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:45.971 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:45.979 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:46.069 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:46.068+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:46.083 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.083 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.091 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:46.170 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:46.169+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:46.235 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.235 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:46.288 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:46.287+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:46.347 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.347 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.356 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:46.371 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:46.371+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:46.472 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:46.471+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:46.499 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.499 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.507 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:46.572 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:46.572+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:46.652 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.652 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.660 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:46.704 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:46.703+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:46.775 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:46.773+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:46.775 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.775 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:46.876 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:46.875+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:46.890 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:46.935 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:46.935 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:46.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:46.985 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:46.984+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:47.010 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:47.010 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:47.027 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:47.027 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:47.043 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:47.043 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:47.059 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:47.059 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:47.075 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:47.075 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:47.077 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:47.076+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:47.087 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:47.087 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:47.091 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:47.091 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:47.096 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:47.108 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:47.108 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:47.125 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:47.126 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:47.178 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:47.178+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:47.200 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.200 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:47.200 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:47.201 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:47.209 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:47.279 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:47.278+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:47.312 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.312 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:47.312 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:47.312 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:47.312 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:47.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:47.405 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:47.404+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.741151501, 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:09:47.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:09:47.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159752 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 327680~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.739651+0000", 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.724181422, 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052236679000000001, 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159752 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739651+0000", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739651+0000", 2024-09-18T01:09:47.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739651+0000", 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9900000000000001e-07 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739660+0000", 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5119999999999997e-06 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739667+0000", 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6610000000000003e-06 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739673+0000", 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2199999999999997e-06 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.791418+0000", 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051745014999999998 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.791457+0000", 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9008000000000003e-05 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.791887+0000", 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042986400000000002 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159753 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 524288~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.739683+0000", 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7241490289999999, 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052821065, 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159753 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739683+0000", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739683+0000", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739683+0000", 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739686+0000", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.424e-06 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739694+0000", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1510000000000004e-06 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739701+0000", 2024-09-18T01:09:47.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7150000000000001e-06 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792091+0000", 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052390758000000003 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792164+0000", 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2469e-05 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792504+0000", 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034006299999999999 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159754 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 720896~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.739709+0000", 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7241231620000002, 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053085462999999999, 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159754 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739709+0000", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739709+0000", 2024-09-18T01:09:47.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739709+0000", 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7800000000000002e-07 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739711+0000", 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2800000000000002e-06 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739719+0000", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7340000000000008e-06 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739725+0000", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4070000000000001e-06 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792538+0000", 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052813457000000001 2024-09-18T01:09:47.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792582+0000", 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748999999999997e-05 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792794+0000", 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000212458 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159755 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 917504~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.739740+0000", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7240922840000001, 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053305999, 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159755 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739740+0000", 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739740+0000", 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739740+0000", 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739743+0000", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5660000000000001e-06 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739751+0000", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5220000000000007e-06 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739757+0000", 2024-09-18T01:09:47.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6160000000000001e-06 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792821+0000", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053063921999999999 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.792865+0000", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4378000000000001e-05 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793046+0000", 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018049 2024-09-18T01:09:47.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159785 3.17 3:edfdc235:::rbd_data.117665245fc1.0000000000000000:42 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.739998+0000", 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7238336510000001, 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053314126000000003, 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159785 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739998+0000", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739998+0000", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.739999+0000", 2024-09-18T01:09:47.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.05e-07 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740000+0000", 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.751e-06 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740006+0000", 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740010+0000", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3370000000000001e-06 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793068+0000", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053058388999999997 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793109+0000", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0244999999999998e-05 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793312+0000", 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020378900000000001 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159786 3.17 3:edfdc235:::rbd_data.117665245fc1.0000000000000000:42 [read 344064~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740016+0000", 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7238156619999998, 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053676808999999999, 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159786 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740016+0000", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740016+0000", 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740017+0000", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740019+0000", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.26e-06 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740025+0000", 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:09:47.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740030+0000", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5800000000000002e-06 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793348+0000", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053318327999999998 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793438+0000", 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0326000000000002e-05 2024-09-18T01:09:47.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793693+0000", 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025481199999999999 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159787 3.17 3:edfdc235:::rbd_data.117665245fc1.0000000000000000:42 [read 540672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740036+0000", 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7237959910000002, 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053966343, 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159787 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740036+0000", 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740036+0000", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740036+0000", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0899999999999997e-07 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740038+0000", 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1059999999999998e-06 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740045+0000", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.285e-06 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740050+0000", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793716+0000", 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053666197999999998 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.793760+0000", 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4189e-05 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794002+0000", 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000241993 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159788 3.17 3:edfdc235:::rbd_data.117665245fc1.0000000000000000:42 [read 737280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740056+0000", 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.723775668, 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054386811, 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159788 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740056+0000", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740056+0000", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740057+0000", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7e-07 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740059+0000", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740065+0000", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4450000000000001e-06 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740070+0000", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2020000000000003e-06 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794042+0000", 2024-09-18T01:09:47.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053971420999999999 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794132+0000", 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0339000000000004e-05 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794443+0000", 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031091400000000003 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159789 3.17 3:edfdc235:::rbd_data.117665245fc1.0000000000000000:42 [read 933888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740076+0000", 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7237557560000001, 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054825803999999999, 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159789 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740076+0000", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740076+0000", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740077+0000", 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:47.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740079+0000", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740085+0000", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.319e-06 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740090+0000", 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0050000000000004e-06 2024-09-18T01:09:47.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794511+0000", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054421084000000002 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794590+0000", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9172999999999996e-05 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794902+0000", 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031171299999999997 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159800 3.1f 3:fb81b21d:::rbd_data.117665245fc1.0000000000000003:42 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740096+0000", 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7237356930000001, 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054776024999999999, 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159800 2024-09-18T01:09:47.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740096+0000", 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740096+0000", 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740097+0000", 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740099+0000", 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1339999999999998e-06 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740106+0000", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740111+0000", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794479+0000", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054368100000000003 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794560+0000", 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0599000000000005e-05 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794872+0000", 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031231299999999999 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159801 3.1f 3:fb81b21d:::rbd_data.117665245fc1.0000000000000003:42 [read 344064~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740118+0000", 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7237140389999999, 2024-09-18T01:09:47.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055161935000000002, 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159801 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740118+0000", 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740118+0000", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740118+0000", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740121+0000", 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740129+0000", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5539999999999998e-06 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740135+0000", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794900+0000", 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054765717999999998 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.794981+0000", 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0721000000000002e-05 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795280+0000", 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029881999999999999 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159802 3.1f 3:fb81b21d:::rbd_data.117665245fc1.0000000000000003:42 [read 540672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740142+0000", 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.723689968, 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055550298999999997, 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159802 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740142+0000", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740142+0000", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740142+0000", 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740145+0000", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740155+0000", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0071000000000001e-05 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740160+0000", 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3909999999999997e-06 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795329+0000", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055169107000000002 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795432+0000", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795692+0000", 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026036499999999998 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159803 3.1f 3:fb81b21d:::rbd_data.117665245fc1.0000000000000003:42 [read 737280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740169+0000", 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.723663132, 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055918830000000003, 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159803 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.525 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740169+0000", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740169+0000", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740169+0000", 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740172+0000", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3439999999999999e-06 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740178+0000", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740183+0000", 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9849999999999997e-06 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795740+0000", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055556856000000002 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.795827+0000", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7569000000000001e-05 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796088+0000", 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026019200000000002 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.528 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159804 3.1f 3:fb81b21d:::rbd_data.117665245fc1.0000000000000003:42 [read 933888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740190+0000", 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7236421589999997, 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.056220622999999997, 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159804 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740190+0000", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740190+0000", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740190+0000", 2024-09-18T01:09:47.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740192+0000", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2230000000000001e-06 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740199+0000", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4949999999999996e-06 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740204+0000", 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.724e-06 2024-09-18T01:09:47.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796132+0000", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055928510000000001 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796182+0000", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9353e-05 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796410+0000", 2024-09-18T01:09:47.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022891200000000001 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159815 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740211+0000", 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.723621149, 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.056290601000000003, 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159815 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740211+0000", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740211+0000", 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740211+0000", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740218+0000", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2230000000000001e-06 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740225+0000", 2024-09-18T01:09:47.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740230+0000", 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1259999999999997e-06 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796127+0000", 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.055896201 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796210+0000", 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3807999999999994e-05 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796501+0000", 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029104700000000001 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159816 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 344064~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740237+0000", 2024-09-18T01:09:47.537 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7235945620000002, 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.056785914999999999, 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159816 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740237+0000", 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740237+0000", 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740238+0000", 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7000000000000005e-07 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740240+0000", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3159999999999999e-06 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740247+0000", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2489999999999998e-06 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740251+0000", 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:09:47.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796532+0000", 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.056280424000000003 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.796609+0000", 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7294999999999998e-05 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797023+0000", 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00041424499999999999 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159817 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 540672~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740256+0000", 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.723575785, 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.057122531999999997, 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159817 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740256+0000", 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740256+0000", 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740257+0000", 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740258+0000", 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7710000000000001e-06 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740262+0000", 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.506e-06 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740264+0000", 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6060000000000001e-06 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797048+0000", 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.056783917000000003 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797112+0000", 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3696999999999997e-05 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797379+0000", 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026664700000000001 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159818 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 737280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740267+0000", 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7235645900000001, 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.057499584999999999, 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159818 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740267+0000", 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740267+0000", 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740268+0000", 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740269+0000", 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.268e-06 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740272+0000", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1389999999999999e-06 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740275+0000", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5170000000000002e-06 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797421+0000", 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.057146777000000003 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797503+0000", 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2070000000000005e-05 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797767+0000", 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026360499999999998 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:159819 3.17 3:eb6c1f59:::rbd_data.117665245fc1.0000000000000006:42 [read 933888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.740277+0000", 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7235546159999999, 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.057748664999999998, 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 159819 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740277+0000", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740277+0000", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740278+0000", 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9400000000000001e-07 2024-09-18T01:09:47.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740279+0000", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.229e-06 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740282+0000", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0860000000000002e-06 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.740285+0000", 2024-09-18T01:09:47.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7709999999999998e-06 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797785+0000", 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.057499898000000001 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.797831+0000", 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6143999999999997e-05 2024-09-18T01:09:47.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.798026+0000", 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000195243 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:47.556 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:47.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:47.558 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:47.557+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:47.659 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:47.658+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:47.701 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:47.702 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:47.702 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:47.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:47.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:09:47.759 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:47.758+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:47.860 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:47.859+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:47.863 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.864 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:47.864 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:47.864 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:47.864 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:47.870 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-18T01:09:47.874 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:47.961 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:47.960+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:47.997 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 132.81730379199999, 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:47.998 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:47.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:48.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:48.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 132.80852184299999, 2024-09-18T01:09:48.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:48.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:48.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:48.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218996912999998, 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:48.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:48.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218974611, 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:48.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:48.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218953657, 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:48.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218932813999999, 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.024 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218911411999997, 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218892232000002, 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218872877000003, 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218834272999999, 2024-09-18T01:09:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218804134000003, 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218780680999998, 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218758665000003, 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218737697000002, 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218716839000002, 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218688389999997, 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218666249000002, 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218642893999998, 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.218619959999998, 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.215580961999997, 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.092 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.094 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:48.094 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:48.093+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:48.147 INFO:teuthology.orchestra.run.smithi120.stdout:3577 punch from 0x9f79cd3 to 0x9f7f7c8, (0x5af5 bytes) 2024-09-18T01:09:48.161 INFO:teuthology.orchestra.run.smithi120.stdout:3579 writesame 0xc974bc7 thru 0xc98150e data_size 0x39(0xc948 bytes) 2024-09-18T01:09:48.161 INFO:teuthology.orchestra.run.smithi120.stdout:3580 clone 34 order 18 su 8192 sc 9 2024-09-18T01:09:48.194 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:48.193+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:48.200 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:48.200 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:48.200 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:48.200 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.209 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:48.295 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:48.294+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:48.351 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:48.351 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.359 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:48.401 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:48.400+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:48.497 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:48.496+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:48.504 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.841133686999999, 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:48.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:09:48.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.841079462, 2024-09-18T01:09:48.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:09:48.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:09:48.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.841047004, 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:09:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:09:48.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.840915741, 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.520 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:09:48.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.840445754999999, 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.525 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:09:48.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.839283155, 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:09:48.529 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:09:48.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:09:48.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:48.534 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.839175873, 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:09:48.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:48.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:09:48.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.839094213999999, 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:09:48.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:09:48.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:09:48.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:09:48.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.838578825999999, 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:09:48.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.837759588000001, 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:09:48.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:48.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.2644945019999998, 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:48.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:09:48.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.563 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158932 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.106905158, 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073292979999999997, 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158932 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395043+0000", 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-07 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395045+0000", 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2019999999999998e-06 2024-09-18T01:09:48.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395048+0000", 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.095e-06 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395051+0000", 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.661e-06 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400622+0000", 2024-09-18T01:09:48.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055717459999999998 2024-09-18T01:09:48.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400709+0000", 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6984999999999995e-05 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401039+0000", 2024-09-18T01:09:48.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032971600000000003 2024-09-18T01:09:48.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401665+0000", 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062626499999999998 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402286+0000", 2024-09-18T01:09:48.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062092099999999997 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402337+0000", 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099e-05 2024-09-18T01:09:48.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402372+0000", 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.434e-05 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.576 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.577 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.577 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0976356100000002, 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.578 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:48.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:09:48.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:09:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.069790222, 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:09:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:09:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.069769923, 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:09:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:09:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:09:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:09:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0697496600000003, 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:09:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:09:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:09:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0697298069999999, 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:09:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:09:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:09:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:09:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.0697106500000002, 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6857202239999998, 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:09:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6257613610000003, 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:09:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:09:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:09:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:48.638 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:48.640 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:48.639+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:48.740 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:48.739+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:48.756 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:09:48.793 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:48.794 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:48.794 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:48.794 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.802 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:48.841 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:48.840+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:48.942 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:48.941+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:48.945 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:48.946 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:48.954 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:49.042 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:49.042+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:49.097 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.468045959999998, 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.468020975000002, 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.468001196000003, 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467974794, 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467953297999998, 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467934002, 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467916168000002, 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467890105000002, 2024-09-18T01:09:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467870750000003, 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467840506000002, 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467821911000001, 2024-09-18T01:09:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467803351999997, 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467785298000003, 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467767539, 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467741269999998, 2024-09-18T01:09:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467722649999999, 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467704400000002, 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.467153025000002, 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.465554038999997, 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.461788052000003, 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:49.184 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:49.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:49.186 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:49.186 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:49.185+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:49.287 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:49.286+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:49.299 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:49.299 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:49.299 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:49.299 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:49.308 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:49.344 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:49.345 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:49.361 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:49.361 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:49.377 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:49.377 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:49.387 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:49.386+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:49.392 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:49.393 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:49.409 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:49.409 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:49.427 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:49.427 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:49.443 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:49.443 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:49.454 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:49.454 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:49.459 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:49.459 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:49.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:49.511 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:49.510+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:49.569 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:49.569 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:49.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:49.623 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:49.622+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:49.690 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:49.689+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:49.707 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:49.707 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:49.717 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:49.801 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:49.800+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:49.893 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:49.891+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:49.911 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:49.911 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:49.923 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:49.994 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:49.993+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:50.095 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:50.094+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:50.103 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.103 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.113 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:50.196 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:50.195+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:50.247 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.247 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.259 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:50.262 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:09:50.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:09:50.307 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:50.306+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:50.362 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.363 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:50.398 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:50.397+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:50.474 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.474 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.483 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:50.527 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:50.526+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:50.586 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.587 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:50.637 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:50.636+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:50.701 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:50.700+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:50.739 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.739 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:50.760 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:09:50.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:09:50.776 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:09:50.818 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:50.817+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:50.854 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:50.854 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:50.863 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:50.903 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-18T01:09:50.904 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:50.903+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:51.003 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:51.002+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:51.004 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:51.005 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:51.014 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:51.104 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:51.104+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout:3581 read 0x5d0871e thru 0x5d159f0 (0xd2d3 bytes) 2024-09-18T01:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout:3583 trunc from 0xc98150f to 0xdf3d3b8 2024-09-18T01:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout:3585 read 0xb704403 thru 0xb70d890 (0x948e bytes) 2024-09-18T01:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout:3587 read 0x7bb2a8d thru 0x7bba698 (0x7c0c bytes) 2024-09-18T01:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout:3589 punch from 0xc605be8 to 0xc60d48f, (0x78a7 bytes) 2024-09-18T01:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout:3590 trunc from 0xdf3d3b8 to 0xcd6f656 2024-09-18T01:09:51.126 INFO:teuthology.orchestra.run.smithi120.stdout:3593 write 0x57f2d24 thru 0x57fed6e (0xc04b bytes) 2024-09-18T01:09:51.131 INFO:teuthology.orchestra.run.smithi120.stdout:3594 read 0x8b77b8c thru 0x8b77f29 (0x39e bytes) 2024-09-18T01:09:51.131 INFO:teuthology.orchestra.run.smithi120.stdout:3595 read 0xa8986d9 thru 0xa8a5e3f (0xd767 bytes) 2024-09-18T01:09:51.132 INFO:teuthology.orchestra.run.smithi120.stdout:3596 write 0x47a5205 thru 0x47b216d (0xcf69 bytes) 2024-09-18T01:09:51.138 INFO:teuthology.orchestra.run.smithi120.stdout:3597 read 0x32c021e thru 0x32cd2d1 (0xd0b4 bytes) 2024-09-18T01:09:51.139 INFO:teuthology.orchestra.run.smithi120.stdout:3598 read 0x6a2dbf5 thru 0x6a2f0b4 (0x14c0 bytes) 2024-09-18T01:09:51.139 INFO:teuthology.orchestra.run.smithi120.stdout:3599 writesame 0x4b87e3a thru 0x4b8f368 data_size 0x1(0x752f bytes) 2024-09-18T01:09:51.153 INFO:teuthology.orchestra.run.smithi120.stdout:3602 read 0xa847f7c thru 0xa84b6f3 (0x3778 bytes) 2024-09-18T01:09:51.153 INFO:teuthology.orchestra.run.smithi120.stdout:3604 read 0x55a3c92 thru 0x55a9e63 (0x61d2 bytes) 2024-09-18T01:09:51.153 INFO:teuthology.orchestra.run.smithi120.stdout:3606 trunc from 0xcd6f656 to 0xd5537f7 2024-09-18T01:09:51.159 INFO:teuthology.orchestra.run.smithi120.stdout:3609 writesame 0xaf60e31 thru 0xaf69e67 data_size 0x1(0x9037 bytes) 2024-09-18T01:09:51.169 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:51.169 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:51.174 INFO:teuthology.orchestra.run.smithi120.stdout:3610 read 0xc03fe80 thru 0xc04722f (0x73b0 bytes) 2024-09-18T01:09:51.174 INFO:teuthology.orchestra.run.smithi120.stdout:3611 read 0xd38a947 thru 0xd392dfa (0x84b4 bytes) 2024-09-18T01:09:51.175 INFO:teuthology.orchestra.run.smithi120.stdout:3612 read 0x196da72 thru 0x19708df (0x2e6e bytes) 2024-09-18T01:09:51.175 INFO:teuthology.orchestra.run.smithi120.stdout:3613 read 0x82019e1 thru 0x82049d0 (0x2ff0 bytes) 2024-09-18T01:09:51.175 INFO:teuthology.orchestra.run.smithi120.stdout:3615 read 0x799ab2f thru 0x79a8e77 (0xe349 bytes) 2024-09-18T01:09:51.176 INFO:teuthology.orchestra.run.smithi120.stdout:3616 trunc from 0xd5537f7 to 0xd808b28 2024-09-18T01:09:51.180 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:51.181 INFO:teuthology.orchestra.run.smithi120.stdout:3617 write 0x315f087 thru 0x316211f (0x3099 bytes) 2024-09-18T01:09:51.183 INFO:teuthology.orchestra.run.smithi120.stdout:3618 punch from 0x14f77cb to 0x14f9de9, (0x261e bytes) 2024-09-18T01:09:51.184 INFO:teuthology.orchestra.run.smithi120.stdout:3619 punch from 0xc4c5cb3 to 0xc4c7e45, (0x2192 bytes) 2024-09-18T01:09:51.190 INFO:teuthology.orchestra.run.smithi120.stdout:3620 write 0xea514f0 thru 0xea591c3 (0x7cd4 bytes) 2024-09-18T01:09:51.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.201 INFO:teuthology.orchestra.run.smithi120.stdout:3623 write 0xc0dda0 thru 0xc1c471 (0xe6d2 bytes) 2024-09-18T01:09:51.205 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:51.205+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:51.208 INFO:teuthology.orchestra.run.smithi120.stdout:3625 writesame 0xdade3d9 thru 0xdae55cb data_size 0x1(0x71f3 bytes) 2024-09-18T01:09:51.223 INFO:teuthology.orchestra.run.smithi120.stdout:3626 read 0xb37f951 thru 0xb383744 (0x3df4 bytes) 2024-09-18T01:09:51.223 INFO:teuthology.orchestra.run.smithi120.stdout:3627 write 0xe21069e thru 0xe210e8a (0x7ed bytes) 2024-09-18T01:09:51.226 INFO:teuthology.orchestra.run.smithi120.stdout:3628 read 0x74a6967 thru 0x74b5b8d (0xf227 bytes) 2024-09-18T01:09:51.239 INFO:teuthology.orchestra.run.smithi120.stdout:3629 trunc from 0xea591c4 to 0x4704025 2024-09-18T01:09:51.298 INFO:teuthology.orchestra.run.smithi120.stdout:3630 write 0x1dc2eb7 thru 0x1dcc33b (0x9485 bytes) 2024-09-18T01:09:51.302 INFO:teuthology.orchestra.run.smithi120.stdout:3631 read 0x272a2a2 thru 0x27305ca (0x6329 bytes) 2024-09-18T01:09:51.302 INFO:teuthology.orchestra.run.smithi120.stdout:3632 write 0xbd35c1 thru 0xbd9ba2 (0x65e2 bytes) 2024-09-18T01:09:51.305 INFO:teuthology.orchestra.run.smithi120.stdout:3633 read 0x2b66c0d thru 0x2b750cb (0xe4bf bytes) 2024-09-18T01:09:51.307 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:51.306+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:51.307 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:51.307 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:51.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:51.324 INFO:teuthology.orchestra.run.smithi120.stdout:3634 write 0xe73e9a9 thru 0xe7435f2 (0x4c4a bytes) 2024-09-18T01:09:51.331 INFO:teuthology.orchestra.run.smithi120.stdout:3636 write 0x58b3e11 thru 0x58c377c (0xf96c bytes) 2024-09-18T01:09:51.349 INFO:teuthology.orchestra.run.smithi120.stdout:3637 trunc from 0xe7435f3 to 0x9e2e9e0 2024-09-18T01:09:51.371 INFO:teuthology.orchestra.run.smithi120.stdout:3638 write 0xd8e39b8 thru 0xd8ebf5d (0x85a6 bytes) 2024-09-18T01:09:51.379 INFO:teuthology.orchestra.run.smithi120.stdout:3639 read 0x64c60a4 thru 0x64cb859 (0x57b6 bytes) 2024-09-18T01:09:51.379 INFO:teuthology.orchestra.run.smithi120.stdout:3640 read 0x51198a4 thru 0x511d7b0 (0x3f0d bytes) 2024-09-18T01:09:51.379 INFO:teuthology.orchestra.run.smithi120.stdout:3641 write 0x1cfede thru 0x1d6a95 (0x6bb8 bytes) 2024-09-18T01:09:51.388 INFO:teuthology.orchestra.run.smithi120.stdout:3642 write 0xe1e33dd thru 0xe1e74a4 (0x40c8 bytes) 2024-09-18T01:09:51.394 INFO:teuthology.orchestra.run.smithi120.stdout:3643 write 0xd64a9a5 thru 0xd65575f (0xadbb bytes) 2024-09-18T01:09:51.404 INFO:teuthology.orchestra.run.smithi120.stdout:3644 writesame 0xb85fc95 thru 0xb861059 data_size 0x7(0x13c5 bytes) 2024-09-18T01:09:51.408 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:51.407+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:51.420 INFO:teuthology.orchestra.run.smithi120.stdout:3645 trunc from 0xe1e74a5 to 0x3d5a484 2024-09-18T01:09:51.444 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:51.445 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:51.455 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:51.455 INFO:teuthology.orchestra.run.smithi120.stdout:truncating to largest ever: 0xee5813e 2024-09-18T01:09:51.468 INFO:teuthology.orchestra.run.smithi120.stdout:3646 trunc from 0x3d5a484 to 0xee5813e 2024-09-18T01:09:51.471 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:51.487 INFO:teuthology.orchestra.run.smithi120.stdout:3648 trunc from 0xee5813e to 0x441ec09 2024-09-18T01:09:51.493 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.493 INFO:teuthology.orchestra.run.smithi120.stdout:3651 read 0x314fbc2 thru 0x315f9d7 (0xfe16 bytes) 2024-09-18T01:09:51.494 INFO:teuthology.orchestra.run.smithi120.stdout:3653 write 0xdfa976 thru 0xe043fd (0x9a88 bytes) 2024-09-18T01:09:51.508 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:51.508+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:51.516 INFO:teuthology.orchestra.run.smithi120.stdout:3654 write 0xad78938 thru 0xad83edc (0xb5a5 bytes) 2024-09-18T01:09:51.525 INFO:teuthology.orchestra.run.smithi120.stdout:3656 write 0x5628f95 thru 0x562c4a1 (0x350d bytes) 2024-09-18T01:09:51.535 INFO:teuthology.orchestra.run.smithi120.stdout:3658 write 0x46e84a thru 0x47df73 (0xf72a bytes) 2024-09-18T01:09:51.541 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.541 INFO:teuthology.orchestra.run.smithi120.stdout:3660 write 0x927692c thru 0x9278ef0 (0x25c5 bytes) 2024-09-18T01:09:51.545 INFO:teuthology.orchestra.run.smithi120.stdout:3661 write 0x105e64e thru 0x1066df8 (0x87ab bytes) 2024-09-18T01:09:51.559 INFO:teuthology.orchestra.run.smithi120.stdout:3663 trunc from 0xad83edd to 0x9ac4306 2024-09-18T01:09:51.574 INFO:teuthology.orchestra.run.smithi120.stdout:3664 write 0x92f23cb thru 0x92f4943 (0x2579 bytes) 2024-09-18T01:09:51.578 INFO:teuthology.orchestra.run.smithi120.stdout:3665 write 0x83d3a8f thru 0x83e19ac (0xdf1e bytes) 2024-09-18T01:09:51.587 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:51.587 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:51.589 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.589 INFO:teuthology.orchestra.run.smithi120.stdout:3668 read 0x9a0de06 thru 0x9a1b4e0 (0xd6db bytes) 2024-09-18T01:09:51.590 INFO:teuthology.orchestra.run.smithi120.stdout:3669 write 0x3a7908a thru 0x3a797f6 (0x76d bytes) 2024-09-18T01:09:51.593 INFO:teuthology.orchestra.run.smithi120.stdout:3671 punch from 0x417294d to 0x417bed4, (0x9587 bytes) 2024-09-18T01:09:51.596 INFO:teuthology.orchestra.run.smithi120.stdout:3672 trunc from 0x9ac4306 to 0x77e73ee 2024-09-18T01:09:51.600 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:51.610 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:51.607+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:51.618 INFO:teuthology.orchestra.run.smithi120.stdout:3673 trunc from 0x77e73ee to 0x340e6df 2024-09-18T01:09:51.624 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:51.624 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:51.642 INFO:teuthology.orchestra.run.smithi120.stdout:3674 trunc from 0x340e6df to 0x5257e0b 2024-09-18T01:09:51.646 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:51.646 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:51.657 INFO:teuthology.orchestra.run.smithi120.stdout:3675 write 0x8124633 thru 0x8127313 (0x2ce1 bytes) 2024-09-18T01:09:51.664 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:51.664 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:51.680 INFO:teuthology.orchestra.run.smithi120.stdout:3676 write 0xcb08530 thru 0xcb11d56 (0x9827 bytes) 2024-09-18T01:09:51.681 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:51.692 INFO:teuthology.orchestra.run.smithi120.stdout:3677 writesame 0x1efda68 thru 0x1efefa4 data_size 0x1(0x153d bytes) 2024-09-18T01:09:51.697 INFO:teuthology.orchestra.run.smithi120.stdout:3678 write 0x370fa06 thru 0x371d050 (0xd64b bytes) 2024-09-18T01:09:51.700 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:51.701 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:51.710 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:51.709+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:51.719 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:51.721 INFO:teuthology.orchestra.run.smithi120.stdout:3679 write 0xd7b1693 thru 0xd7b2298 (0xc06 bytes) 2024-09-18T01:09:51.729 INFO:teuthology.orchestra.run.smithi120.stdout:3680 read 0x4442e7a thru 0x444b466 (0x85ed bytes) 2024-09-18T01:09:51.729 INFO:teuthology.orchestra.run.smithi120.stdout:3681 punch from 0x39aa738 to 0x39b120d, (0x6ad5 bytes) 2024-09-18T01:09:51.729 INFO:teuthology.orchestra.run.smithi120.stdout:3682 read 0x668d605 thru 0x6691a5d (0x4459 bytes) 2024-09-18T01:09:51.730 INFO:teuthology.orchestra.run.smithi120.stdout:3684 write 0x242e2db thru 0x242f75e (0x1484 bytes) 2024-09-18T01:09:51.730 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.730 INFO:teuthology.orchestra.run.smithi120.stdout:3690 write 0x4893cd6 thru 0x4894408 (0x733 bytes) 2024-09-18T01:09:51.735 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2196 3.4 3:26396459:::rbd_data.11769efc726f.000000000000003e:head [write 0~376832 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 136.55450385099999, 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032729037000000002, 2024-09-18T01:09:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2196 2024-09-18T01:09:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178515+0000", 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178516+0000", 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178630+0000", 2024-09-18T01:09:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114468 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178640+0000", 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8840000000000006e-06 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.178645+0000", 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7910000000000004e-06 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.205859+0000", 2024-09-18T01:09:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.027214087000000001 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206001+0000", 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014199199999999999 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.206158+0000", 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000156859 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.210014+0000", 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003856029 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211180+0000", 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001165835 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211209+0000", 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8855000000000001e-05 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.211244+0000", 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5791999999999999e-05 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:2237 3.4 3:26053817:::rbd_data.11769efc726f.0000000000000043:head [write 0~16384 [fadvise_sequential+fadvise_nocache],write 524288~458752 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 136.545721902, 2024-09-18T01:09:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032564693999999998, 2024-09-18T01:09:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 2237 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187297+0000", 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187299+0000", 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.466e-06 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187631+0000", 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00033183099999999999 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187643+0000", 2024-09-18T01:09:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2636999999999999e-05 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.187666+0000", 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3040999999999998e-05 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207423+0000", 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019756672999999999 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207572+0000", 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149037 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.207788+0000", 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021613599999999999 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.216014+0000", 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008225497 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219758+0000", 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003743878 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219810+0000", 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2021999999999998e-05 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:35.219862+0000", 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475999999999997e-05 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68332 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 667648~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956196972000001, 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029580842999999999, 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.756 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68332 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776822+0000", 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776823+0000", 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776825+0000", 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3760000000000002e-06 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776833+0000", 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0579999999999998e-06 2024-09-18T01:09:51.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776838+0000", 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806290+0000", 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029451779000000001 2024-09-18T01:09:51.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806347+0000", 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6838e-05 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806403+0000", 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6165999999999997e-05 2024-09-18T01:09:51.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68333 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 690176~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956174670000003, 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029748068999999999, 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68333 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776845+0000", 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.27e-07 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776847+0000", 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0829999999999998e-06 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776854+0000", 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2269999999999996e-06 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776859+0000", 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8509999999999998e-06 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806431+0000", 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029572136999999998 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806508+0000", 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806593+0000", 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4926999999999995e-05 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68334 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 712704~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956153716000003, 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029873443, 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68334 2024-09-18T01:09:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776866+0000", 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776868+0000", 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.024e-06 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776876+0000", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6249999999999998e-06 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776880+0000", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.69e-06 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806623+0000", 2024-09-18T01:09:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029742273 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806677+0000", 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4128000000000001e-05 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806739+0000", 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2317000000000007e-05 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68335 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 735232~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956132873000001, 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.03005594, 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68335 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776886+0000", 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776887+0000", 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776889+0000", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776895+0000", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776902+0000", 2024-09-18T01:09:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3790000000000003e-06 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806766+0000", 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029864695 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806840+0000", 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3345999999999999e-05 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806942+0000", 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010265299999999999 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68336 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 757760~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956111471, 2024-09-18T01:09:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030175579000000001, 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68336 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776908+0000", 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776910+0000", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8199999999999999e-06 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776916+0000", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1770000000000003e-06 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776921+0000", 2024-09-18T01:09:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8940000000000001e-06 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.806973+0000", 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030051451 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807026+0000", 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807083+0000", 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7228000000000002e-05 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68337 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 780288~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956092290999997, 2024-09-18T01:09:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030395874, 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68337 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776927+0000", 2024-09-18T01:09:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776929+0000", 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9130000000000001e-06 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776936+0000", 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5370000000000003e-06 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776941+0000", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807130+0000", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030189015 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807214+0000", 2024-09-18T01:09:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4773999999999998e-05 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807323+0000", 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108379 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68338 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 802816~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956072935999998, 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030646049000000002, 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68338 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776946+0000", 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776947+0000", 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776955+0000", 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776963+0000", 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776969+0000", 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807422+0000", 2024-09-18T01:09:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030453634 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807502+0000", 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9957999999999996e-05 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807592+0000", 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0265000000000004e-05 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68339 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 825344~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956034332000002, 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030841727999999999, 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68339 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776985+0000", 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776989+0000", 2024-09-18T01:09:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9519999999999996e-06 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.776999+0000", 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1959999999999997e-06 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777006+0000", 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1969999999999999e-06 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807630+0000", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030624551999999999 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807713+0000", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3203999999999998e-05 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807827+0000", 2024-09-18T01:09:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011317200000000001 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68340 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 847872~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.956004192999998, 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031032458999999998, 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68340 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777015+0000", 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777016+0000", 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7399999999999998e-07 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777018+0000", 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777027+0000", 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0890000000000006e-06 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777032+0000", 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807859+0000", 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030827113 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.807937+0000", 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8009999999999993e-05 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808048+0000", 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011049599999999999 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68341 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 870400~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955980740000001, 2024-09-18T01:09:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031214333, 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68341 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777039+0000", 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777041+0000", 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777049+0000", 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6450000000000005e-06 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777054+0000", 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0220000000000004e-06 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808077+0000", 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031023016 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808156+0000", 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8961000000000005e-05 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808253+0000", 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7252999999999999e-05 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68342 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 892928~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955958723999998, 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031397033999999997, 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68342 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777061+0000", 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777063+0000", 2024-09-18T01:09:51.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777070+0000", 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5429999999999996e-06 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777075+0000", 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.775e-06 2024-09-18T01:09:51.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808285+0000", 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031209436 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808360+0000", 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5444000000000005e-05 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808458+0000", 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7529999999999996e-05 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68343 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 915456~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955937755999997, 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031530233999999997, 2024-09-18T01:09:51.815 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68343 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777082+0000", 2024-09-18T01:09:51.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777084+0000", 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8780000000000001e-06 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777091+0000", 2024-09-18T01:09:51.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777096+0000", 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808501+0000", 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031405020999999998 2024-09-18T01:09:51.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808549+0000", 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7957000000000002e-05 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808612+0000", 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2676999999999996e-05 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68344 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 937984~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955916897999998, 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031770988999999999, 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68344 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777102+0000", 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777103+0000", 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777105+0000", 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8169999999999999e-06 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777112+0000", 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6709999999999999e-06 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777118+0000", 2024-09-18T01:09:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1630000000000001e-06 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808651+0000", 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031532683999999998 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808769+0000", 2024-09-18T01:09:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011837600000000001 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808873+0000", 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103896 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68345 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 960512~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955888449, 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031919627999999999, 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.827 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68345 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:51.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777131+0000", 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777133+0000", 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0889999999999998e-06 2024-09-18T01:09:51.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777141+0000", 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777146+0000", 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3449999999999996e-06 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808916+0000", 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031769409999999998 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.808961+0000", 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5451000000000002e-05 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809050+0000", 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9167999999999997e-05 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68346 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 983040~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:51.832 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955866307999997, 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032123031000000003, 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68346 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:51.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777153+0000", 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777155+0000", 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8789999999999999e-06 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777163+0000", 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6199999999999999e-06 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777169+0000", 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2890000000000003e-06 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809090+0000", 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.031921136000000003 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809171+0000", 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.119e-05 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809276+0000", 2024-09-18T01:09:51.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104576 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68347 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1005568~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955842953000001, 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032286843000000003, 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.838 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68347 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777176+0000", 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777177+0000", 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777179+0000", 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.018e-06 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777186+0000", 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6890000000000004e-06 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777193+0000", 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809334+0000", 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032141019 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809377+0000", 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3259e-05 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809463+0000", 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6231999999999997e-05 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68348 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:34 [read 1028096~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.955820019000001, 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032479804000000001, 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68348 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777199+0000", 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777200+0000", 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777202+0000", 2024-09-18T01:09:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8810000000000001e-06 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777209+0000", 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4279999999999999e-06 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.777214+0000", 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-18T01:09:51.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809502+0000", 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.032288490000000003 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809581+0000", 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8452e-05 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809679+0000", 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8220999999999994e-05 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:68366 3.0 3:0098db63:::rbd_data.117658cbc921.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.952781021, 2024-09-18T01:09:51.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029673050999999999, 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 68366 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:51.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780238+0000", 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780239+0000", 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0750000000000001e-06 2024-09-18T01:09:51.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780244+0000", 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6589999999999998e-06 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780256+0000", 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1646e-05 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.780263+0000", 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.503e-06 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809719+0000", 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.029455822 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:51.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809790+0000", 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0629000000000004e-05 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.809911+0000", 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121717 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:51.854 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:51.855 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:51.856 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:51.856 INFO:teuthology.orchestra.run.smithi120.stdout:3691 write 0xe1887a4 thru 0xe192a27 (0xa284 bytes) 2024-09-18T01:09:51.857 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:3692 read 0x705fcd7 thru 0x706d790 (0xdaba bytes) 2024-09-18T01:09:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:3697 punch from 0x918d808 to 0x9191957, (0x414f bytes) 2024-09-18T01:09:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:3698 write 0xf0306d thru 0xf0c395 (0x9329 bytes) 2024-09-18T01:09:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:51.859 INFO:teuthology.orchestra.run.smithi120.stdout:3699 read 0xc65a48e thru 0xc65eeb6 (0x4a29 bytes) 2024-09-18T01:09:51.860 INFO:teuthology.orchestra.run.smithi120.stdout:3701 trunc from 0xe192a28 to 0x5f76ebd 2024-09-18T01:09:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:09:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:09:51.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:09:51.862 INFO:teuthology.orchestra.run.smithi120.stdout:3702 read 0x566ebf1 thru 0x5678a3e (0x9e4e bytes) 2024-09-18T01:09:51.862 INFO:teuthology.orchestra.run.smithi120.stdout:3703 trunc from 0x5f76ebd to 0x6c488d3 2024-09-18T01:09:51.864 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:51.854+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:51.866 INFO:teuthology.orchestra.run.smithi120.stdout:3705 punch from 0x693f28a to 0x694b699, (0xc40f bytes) 2024-09-18T01:09:51.866 INFO:teuthology.orchestra.run.smithi120.stdout:3706 write 0x863ea6f thru 0x86446b8 (0x5c4a bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3707 read 0x3d92741 thru 0x3da12d3 (0xeb93 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3708 read 0x4ea9c0c thru 0x4eb82bd (0xe6b2 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3709 punch from 0x63abb53 to 0x63b05ec, (0x4a99 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3710 read 0x290487f thru 0x29073ec (0x2b6e bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3711 trunc from 0x86446b9 to 0xc9e3958 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3712 punch from 0x2ec75f3 to 0x2ed488a, (0xd297 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3714 punch from 0x2af2a93 to 0x2af60c2, (0x362f bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3715 read 0x6fb4e93 thru 0x6fbb359 (0x64c7 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3716 read 0x690ba18 thru 0x6915837 (0x9e20 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3717 write 0xe8955e thru 0xe90aca (0x756d bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3719 read 0xb7c2a75 thru 0xb7c83a8 (0x5934 bytes) 2024-09-18T01:09:51.867 INFO:teuthology.orchestra.run.smithi120.stdout:3720 read 0x69aa8a8 thru 0x69b914e (0xe8a7 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3721 read 0xb451172 thru 0xb45d7c9 (0xc658 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3722 read 0x1c75fbc thru 0x1c77e85 (0x1eca bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3723 write 0xe12ec68 thru 0xe132e0d (0x41a6 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3725 trunc from 0xe132e0e to 0xe3438ee 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3727 writesame 0x1c7d0bc thru 0x1c84413 data_size 0x4(0x7358 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3728 read 0x8171158 thru 0x817b86a (0xa713 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3729 punch from 0xba01306 to 0xba061af, (0x4ea9 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3731 writesame 0x3402d30 thru 0x3404233 data_size 0xa(0x1504 bytes) 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:51.868 INFO:teuthology.orchestra.run.smithi120.stdout:3733 read 0x37bbfe2 thru 0x37c4288 (0x82a7 bytes) 2024-09-18T01:09:51.869 INFO:teuthology.orchestra.run.smithi120.stdout:3734 read 0xad91a0 thru 0xadf376 (0x61d7 bytes) 2024-09-18T01:09:51.869 INFO:teuthology.orchestra.run.smithi120.stdout:3735 read 0xb96898e thru 0xb97423b (0xb8ae bytes) 2024-09-18T01:09:51.869 INFO:teuthology.orchestra.run.smithi120.stdout:3736 write 0xc1a97fe thru 0xc1afe2e (0x6631 bytes) 2024-09-18T01:09:51.874 INFO:teuthology.orchestra.run.smithi120.stdout:3738 read 0x1172e25 thru 0x118265b (0xf837 bytes) 2024-09-18T01:09:51.874 INFO:teuthology.orchestra.run.smithi120.stdout:3740 writesame 0xc9055a0 thru 0xc909599 data_size 0x13(0x3ffa bytes) 2024-09-18T01:09:51.880 INFO:teuthology.orchestra.run.smithi120.stdout:3741 writesame 0xe09e4d1 thru 0xe0a3852 data_size 0x15(0x5382 bytes) 2024-09-18T01:09:51.888 INFO:teuthology.orchestra.run.smithi120.stdout:3742 read 0x5f8786c thru 0x5f91eef (0xa684 bytes) 2024-09-18T01:09:51.901 INFO:teuthology.orchestra.run.smithi120.stdout:3743 trunc from 0xe3438ee to 0x32f4dba 2024-09-18T01:09:51.931 INFO:teuthology.orchestra.run.smithi120.stdout:3744 read 0xc0f61a thru 0xc1c0af (0xca96 bytes) 2024-09-18T01:09:51.943 INFO:teuthology.orchestra.run.smithi120.stdout:3745 writesame 0x716b83a thru 0x7176e2b data_size 0x15(0xb5f2 bytes) 2024-09-18T01:09:51.943 INFO:teuthology.orchestra.run.smithi120.stdout:3746 read 0x3755abb thru 0x3763e76 (0xe3bc bytes) 2024-09-18T01:09:51.944 INFO:teuthology.orchestra.run.smithi120.stdout:3748 punch from 0xfd8e3a to 0xfdcd1b, (0x3ee1 bytes) 2024-09-18T01:09:51.944 INFO:teuthology.orchestra.run.smithi120.stdout:3749 write 0x4cec4b2 thru 0x4cfa90a (0xe459 bytes) 2024-09-18T01:09:51.957 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:51.956+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:51.959 INFO:teuthology.orchestra.run.smithi120.stdout:3750 punch from 0x22d764 to 0x232e25, (0x56c1 bytes) 2024-09-18T01:09:51.970 INFO:teuthology.orchestra.run.smithi120.stdout:3752 write 0xa62a867 thru 0xa632e8a (0x8624 bytes) 2024-09-18T01:09:51.978 INFO:teuthology.orchestra.run.smithi120.stdout:3753 write 0xa12abf7 thru 0xa1345bf (0x99c9 bytes) 2024-09-18T01:09:51.989 INFO:teuthology.orchestra.run.smithi120.stdout:3754 punch from 0x6b9539d to 0x6b9c329, (0x6f8c bytes) 2024-09-18T01:09:51.990 INFO:teuthology.orchestra.run.smithi120.stdout:3756 read 0x4dd1b23 thru 0x4dd588a (0x3d68 bytes) 2024-09-18T01:09:51.990 INFO:teuthology.orchestra.run.smithi120.stdout:3757 read 0x3ea2fbc thru 0x3ead59e (0xa5e3 bytes) 2024-09-18T01:09:51.990 INFO:teuthology.orchestra.run.smithi120.stdout:3759 write 0x45627da thru 0x456a104 (0x792b bytes) 2024-09-18T01:09:51.991 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:51.992 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.001 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:52.006 INFO:teuthology.orchestra.run.smithi120.stdout:3760 read 0x8ba11ac thru 0x8bab19e (0x9ff3 bytes) 2024-09-18T01:09:52.006 INFO:teuthology.orchestra.run.smithi120.stdout:3762 write 0x1e400d5 thru 0x1e4ad38 (0xac64 bytes) 2024-09-18T01:09:52.011 INFO:teuthology.orchestra.run.smithi120.stdout:3763 punch from 0x90f79d9 to 0x90f9ea3, (0x24ca bytes) 2024-09-18T01:09:52.011 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:09:52.011 INFO:teuthology.orchestra.run.smithi120.stdout:3765 trunc from 0xa632e8b to 0xa76b3e5 2024-09-18T01:09:52.017 INFO:teuthology.orchestra.run.smithi120.stdout:3767 clone 35 order 18 su 1024 sc 14 2024-09-18T01:09:52.058 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:52.057+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:52.112 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:52.112 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:52.112 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:52.113 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:52.113 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.121 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:52.160 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:52.158+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:52.260 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:52.259+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:52.266 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.603556265000002, 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:52.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:09:52.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.271 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.603502039999999, 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:09:52.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.276 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.603469581999999, 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:09:52.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.281 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.603338318999999, 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.286 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.602868333, 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:52.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:09:52.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.291 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.601705732999999, 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:09:52.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:09:52.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.601598451000001, 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.297 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:09:52.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:09:52.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.601516791999998, 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:09:52.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:09:52.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.601001404000002, 2024-09-18T01:09:52.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:09:52.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:09:52.311 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.600182166, 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:09:52.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:09:52.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.02691708, 2024-09-18T01:09:52.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:52.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:09:52.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:09:52.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158932 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8693277360000007, 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073292979999999997, 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158932 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395042+0000", 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395043+0000", 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-07 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395045+0000", 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2019999999999998e-06 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395048+0000", 2024-09-18T01:09:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.095e-06 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.395051+0000", 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.661e-06 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400622+0000", 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055717459999999998 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.400709+0000", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6984999999999995e-05 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401039+0000", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032971600000000003 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.401665+0000", 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062626499999999998 2024-09-18T01:09:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402286+0000", 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00062092099999999997 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402337+0000", 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099e-05 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.402372+0000", 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.434e-05 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.860058188, 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:09:52.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:09:52.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.335 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8322128000000006, 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:09:52.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8321925009999998, 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:09:52.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:09:52.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8321722380000001, 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:09:52.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:09:52.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.356 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8321523850000005, 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:09:52.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:09:52.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.832133228, 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:09:52.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.4481428019999996, 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:09:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:09:52.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:09:52.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:52.375 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.3881839389999993, 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:09:52.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.381 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.383 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:52.383 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:52.382+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:52.484 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:52.483+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:52.532 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:09:52.532 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:52.624 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:52.623+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:52.682 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:52.682 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:09:52.682 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:09:52.683 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:09:52.683 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.686 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:52.685+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:52.691 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:09:52.787 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:52.785+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:52.795 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220420054000002, 2024-09-18T01:09:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:52.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220395068999999, 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:52.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.22037529, 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:52.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220348887999997, 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:52.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:52.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:52.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220327392000002, 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.868 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:52.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:52.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:52.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220308095999997, 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:52.872 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:52.876 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220290261999999, 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.880 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220264198999999, 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:52.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:52.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220244844, 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.885 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220214599999998, 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:52.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220196004999998, 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.894 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:52.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220177446000001, 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:52.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220159391999999, 2024-09-18T01:09:52.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:52.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:52.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220141632999997, 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.910 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220115364000002, 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:52.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220096744000003, 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.220078493999999, 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:52.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.219527118999999, 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:52.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:52.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.217928133000001, 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:52.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:52.933 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.214162146, 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:52.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:52.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:52.938 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:52.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:52.940 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:52.937+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:53.040 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:53.039+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:53.049 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.049 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:53.049 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:53.049 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.058 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:53.141 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:53.139+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:53.162 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.162 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:53.162 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:53.162 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:53.162 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.170 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:53.242 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:53.241+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9170 3.1d e34/23 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head v 34'32, mlcod=29'30)", 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.607480713, 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027766514999999999, 2024-09-18T01:09:53.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664553+0000", 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664554+0000", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667191+0000", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026376149999999998 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667211+0000", 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9579999999999999e-05 2024-09-18T01:09:53.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667221+0000", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.944e-06 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667290+0000", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9263000000000004e-05 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.667311+0000", 2024-09-18T01:09:53.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1087999999999999e-05 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692259+0000", 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024947540000000001 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692320+0000", 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0724999999999999e-05 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.549293134, 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:53.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:53.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:53.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:53.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70097 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1181696~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409797531000002, 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023243085, 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70097 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862236+0000", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862237+0000", 2024-09-18T01:09:53.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862243+0000", 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862248+0000", 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862252+0000", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.707e-06 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885228+0000", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022975553999999999 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885312+0000", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3826999999999995e-05 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885479+0000", 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016766599999999999 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70098 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1204224~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409775011999997, 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023388513, 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70098 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:53.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862259+0000", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2800000000000002e-07 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862261+0000", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.232e-06 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862267+0000", 2024-09-18T01:09:53.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8819999999999998e-06 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862272+0000", 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885516+0000", 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023244317 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885573+0000", 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6752999999999997e-05 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885647+0000", 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4031999999999999e-05 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70099 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1226752~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409753444000003, 2024-09-18T01:09:53.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023532469, 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70099 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862280+0000", 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862281+0000", 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862283+0000", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862290+0000", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862295+0000", 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:53.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885686+0000", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023391835999999999 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885745+0000", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8235000000000001e-05 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885813+0000", 2024-09-18T01:09:53.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70100 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1249280~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409732675000001, 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023768480000000002, 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70100 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862301+0000", 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862302+0000", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862304+0000", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862309+0000", 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:09:53.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862314+0000", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885866+0000", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023551761000000001 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.885962+0000", 2024-09-18T01:09:53.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6531000000000001e-05 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886070+0000", 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107154 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70101 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1271808~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:53.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409713480000001, 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023896830000000001, 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70101 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:53.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862320+0000", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862321+0000", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862323+0000", 2024-09-18T01:09:53.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2409999999999998e-06 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862329+0000", 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6370000000000004e-06 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862333+0000", 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4660000000000001e-06 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886089+0000", 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023756401999999999 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886137+0000", 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7796000000000001e-05 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886217+0000", 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9871999999999997e-05 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70102 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1294336~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409694694000002, 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024103396999999999, 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70102 2024-09-18T01:09:53.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862339+0000", 2024-09-18T01:09:53.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862342+0000", 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862348+0000", 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9449999999999998e-06 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862352+0000", 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4649999999999996e-06 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886254+0000", 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023902041999999998 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886335+0000", 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1138999999999997e-05 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886442+0000", 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010723299999999999 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70103 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1316864~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409675385, 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02429678, 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70103 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862358+0000", 2024-09-18T01:09:53.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862359+0000", 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862361+0000", 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862368+0000", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7830000000000001e-06 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862374+0000", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886473+0000", 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024099222999999999 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886561+0000", 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7099000000000001e-05 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886655+0000", 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4643999999999997e-05 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70104 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1339392~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409652995999998, 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024466301999999999, 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70104 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862381+0000", 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:09:53.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862383+0000", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1260000000000001e-06 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862391+0000", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862396+0000", 2024-09-18T01:09:53.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1490000000000002e-06 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886684+0000", 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024288026000000001 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886758+0000", 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886847+0000", 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8871999999999999e-05 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70105 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1361920~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409631675999997, 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024652534, 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.322 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70105 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862402+0000", 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862405+0000", 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.199e-06 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862411+0000", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862420+0000", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7579999999999998e-06 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886876+0000", 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024456041000000001 2024-09-18T01:09:53.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.886951+0000", 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5640000000000001e-05 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887055+0000", 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103372 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70106 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1384448~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409608083000002, 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024855723999999999, 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70106 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862426+0000", 2024-09-18T01:09:53.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862428+0000", 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1569999999999998e-06 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862435+0000", 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862440+0000", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887087+0000", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024647281 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887167+0000", 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9874000000000001e-05 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887281+0000", 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114471 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70107 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1406976~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409588167000003, 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025067570000000001, 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70107 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862446+0000", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0600000000000001e-07 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862447+0000", 2024-09-18T01:09:53.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.345e-06 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862451+0000", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5099999999999999e-06 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862454+0000", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7110000000000001e-06 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887320+0000", 2024-09-18T01:09:53.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024865986 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887408+0000", 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7573999999999999e-05 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887513+0000", 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105138 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70108 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1429504~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409569736000002, 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025250589, 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.336 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70108 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862464+0000", 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862465+0000", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862466+0000", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.872e-06 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862476+0000", 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.329e-06 2024-09-18T01:09:53.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862481+0000", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2469999999999999e-06 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887544+0000", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025062705000000001 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887621+0000", 2024-09-18T01:09:53.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7836000000000005e-05 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887715+0000", 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3108000000000004e-05 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70109 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1452032~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409545471999998, 2024-09-18T01:09:53.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025363568999999999, 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70109 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862488+0000", 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862489+0000", 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862491+0000", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862500+0000", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2580000000000001e-06 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862506+0000", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4020000000000001e-06 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887744+0000", 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025238536999999998 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887789+0000", 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4867999999999998e-05 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887852+0000", 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2866000000000004e-05 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.344 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70110 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1474560~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409521097999999, 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025498745, 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70110 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862513+0000", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862515+0000", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.08e-06 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862522+0000", 2024-09-18T01:09:53.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4869999999999999e-06 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862527+0000", 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.203e-06 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887884+0000", 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025357273 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.887954+0000", 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9361000000000002e-05 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888011+0000", 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7868999999999998e-05 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70111 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1497088~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.349 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409499957999998, 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025710099, 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70111 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862534+0000", 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:53.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862536+0000", 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862543+0000", 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4420000000000003e-06 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862547+0000", 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888053+0000", 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025505574 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888139+0000", 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5854999999999997e-05 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888244+0000", 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104999 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70112 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1519616~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409479875999999, 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025916913999999999, 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70112 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862554+0000", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862556+0000", 2024-09-18T01:09:53.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862563+0000", 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5359999999999998e-06 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862568+0000", 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888297+0000", 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025729429000000002 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888360+0000", 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2630000000000002e-05 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888471+0000", 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000111075 2024-09-18T01:09:53.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70113 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1542144~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409459730000002, 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026045788, 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70113 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862574+0000", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862576+0000", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862583+0000", 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4169999999999997e-06 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862588+0000", 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1340000000000003e-06 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888506+0000", 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025918170000000001 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888551+0000", 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.456e-05 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888620+0000", 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9142999999999999e-05 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:70114 3.9 3:901cc600:::rbd_data.117658cbc921.0000000000000003:34 [read 1564672~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.409439667000001, 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026267357000000002, 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 70114 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862594+0000", 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:09:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862596+0000", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862603+0000", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.862607+0000", 2024-09-18T01:09:53.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6909999999999996e-06 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888664+0000", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026056645999999999 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888756+0000", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1853000000000003e-05 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.888861+0000", 2024-09-18T01:09:53.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105535 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.368 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.369 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:53.370 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:53.368+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:53.470 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:53.468+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:53.514 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.514 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:53.514 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:53.515 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:53.571 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:53.569+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:53.626 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:53.626 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:53.678 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:53.676+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:649 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71525791299999, 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021686365999999999, 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 649 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:53.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023086+0000", 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023087+0000", 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023095+0000", 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2190000000000004e-06 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023102+0000", 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5010000000000001e-06 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023107+0000", 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044583+0000", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021476111999999999 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044663+0000", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0270000000000002e-05 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044773+0000", 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109756 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:650 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71521576200001, 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021804799999999999, 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 650 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:53.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023129+0000", 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9400000000000005e-07 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023132+0000", 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023139+0000", 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023153+0000", 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4156e-05 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044805+0000", 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021651710000000001 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044854+0000", 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044933+0000", 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8928000000000001e-05 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:651 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1163264~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.749 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71518233399999, 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021992040000000001, 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 651 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:53.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023162+0000", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023163+0000", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023165+0000", 2024-09-18T01:09:53.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7480000000000002e-06 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023172+0000", 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6470000000000001e-06 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023177+0000", 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.044971+0000", 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021793739999999999 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045052+0000", 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1192999999999993e-05 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045154+0000", 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102271 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:652 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1294336~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.715144834, 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022110696999999999, 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 652 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:53.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023200+0000", 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023202+0000", 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023208+0000", 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2559999999999999e-06 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023214+0000", 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0310000000000004e-06 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045184+0000", 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021969890999999998 2024-09-18T01:09:53.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045233+0000", 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8338999999999999e-05 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045310+0000", 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7636e-05 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:653 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1425408~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71512325699999, 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022268056000000001, 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 653 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023221+0000", 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023222+0000", 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023224+0000", 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023229+0000", 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4589999999999997e-06 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023234+0000", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3030000000000001e-06 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045345+0000", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022111873000000001 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045406+0000", 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0742000000000002e-05 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045489+0000", 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2966999999999994e-05 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:654 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1556480~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71510411700001, 2024-09-18T01:09:53.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022430181, 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 654 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:53.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023240+0000", 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023241+0000", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023243+0000", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3199999999999998e-06 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023249+0000", 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:09:53.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023253+0000", 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3379999999999998e-06 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045529+0000", 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022275896999999999 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045584+0000", 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700999999999997e-05 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045670+0000", 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6363e-05 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.768 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:655 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1687552~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71508372599999, 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022551008000000001, 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 655 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023261+0000", 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023263+0000", 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023269+0000", 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7259999999999999e-06 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023273+0000", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5349999999999998e-06 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045702+0000", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022428785999999999 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045750+0000", 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7688999999999997e-05 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045812+0000", 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1736000000000007e-05 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:656 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1818624~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.715054326, 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022678336, 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 656 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023290+0000", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023291+0000", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023293+0000", 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023300+0000", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023311+0000", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045841+0000", 2024-09-18T01:09:53.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022529878 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045887+0000", 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5537999999999997e-05 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045968+0000", 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1859000000000003e-05 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:657 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 1949696~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71502553600001, 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022779777000000001, 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 657 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:53.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023319+0000", 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3e-07 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023322+0000", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4260000000000002e-06 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023328+0000", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000003e-06 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023333+0000", 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:09:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.045995+0000", 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022661361000000001 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046038+0000", 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3430999999999998e-05 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046099+0000", 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0621000000000001e-05 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:658 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2080768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.715004841, 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022890766999999999, 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.783 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 658 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023340+0000", 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023342+0000", 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.086e-06 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023348+0000", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023353+0000", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6739999999999996e-06 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046126+0000", 2024-09-18T01:09:53.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022773748 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046169+0000", 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2622e-05 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046230+0000", 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1192000000000006e-05 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:659 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2211840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71498535699999, 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022996493, 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.788 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 659 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023359+0000", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023373+0000", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3152999999999999e-05 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023380+0000", 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9489999999999999e-06 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023385+0000", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046255+0000", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022870418 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046297+0000", 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-05 2024-09-18T01:09:53.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046356+0000", 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8922999999999998e-05 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:660 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2342912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71495144599999, 2024-09-18T01:09:53.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023103678999999998, 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 660 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023393+0000", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023401+0000", 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.593e-06 2024-09-18T01:09:53.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023408+0000", 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9430000000000003e-06 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023413+0000", 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046383+0000", 2024-09-18T01:09:53.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022970441000000001 2024-09-18T01:09:53.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046428+0000", 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4101999999999999e-05 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046497+0000", 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9090000000000004e-05 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:661 3.16 3:6d91b72a:::rbd_data.1176f5693ee0.0000000000000001:3 [read 2473984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.71491544700001, 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023185339999999999, 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 661 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023429+0000", 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023430+0000", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-07 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023432+0000", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.695e-06 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023439+0000", 2024-09-18T01:09:53.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7619999999999998e-06 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.023444+0000", 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046514+0000", 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023069164999999999 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046556+0000", 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2865999999999999e-05 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.046614+0000", 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7833e-05 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1452 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3260416~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.63046747499999, 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021734323, 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1452 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107877+0000", 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.53e-07 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107878+0000", 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.173e-06 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107882+0000", 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2339999999999999e-06 2024-09-18T01:09:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107884+0000", 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4880000000000001e-06 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129415+0000", 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021530796000000001 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129498+0000", 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3174e-05 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129611+0000", 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113205 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:1453 3.1d 3:bb6ef873:::rbd_data.1176f5693ee0.0000000000000009:3 [read 3391488~1188 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:53.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.63045726600001, 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022072108999999999, 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1453 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107887+0000", 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:53.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107889+0000", 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9489999999999999e-06 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107893+0000", 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999999e-06 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.107895+0000", 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.435e-06 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129646+0000", 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021750693000000001 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129730+0000", 2024-09-18T01:09:53.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4206e-05 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:35.129959+0000", 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228929 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.13005604, 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:53.815 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:53.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:53.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.821 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.11456987099999, 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025968161, 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for ondisk", 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623775+0000", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.623776+0000", 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.776e-06 2024-09-18T01:09:53.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630305+0000", 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065285450000000002 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630322+0000", 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7193999999999998e-05 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630331+0000", 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.568e-06 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630405+0000", 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4878999999999999e-05 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for ondisk", 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.630462+0000", 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6286999999999998e-05 2024-09-18T01:09:53.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649743+0000", 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019280912000000001 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.08061014800001, 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.828 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:53.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:53.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:53.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:53.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4470.0:9168 3.1f e34/23 3:fc6e724c:::rbd_data.11762d3aac1.000000000000000a:head v 34'14, mlcod=31'12)", 2024-09-18T01:09:53.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.075441356, 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028264758000000001, 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662903+0000", 2024-09-18T01:09:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.662904+0000", 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3999999999999999e-06 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670682+0000", 2024-09-18T01:09:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077772600000000002 2024-09-18T01:09:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670702+0000", 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0565999999999999e-05 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670714+0000", 2024-09-18T01:09:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.172e-05 2024-09-18T01:09:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670794+0000", 2024-09-18T01:09:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9598999999999995e-05 2024-09-18T01:09:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.670818+0000", 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4630999999999998e-05 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691094+0000", 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276235 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.691168+0000", 2024-09-18T01:09:53.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3347000000000001e-05 2024-09-18T01:09:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:121600 3.5 3:a5560957:::rbd_header.1176e38d32d8:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.168198497999999, 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034695009999999998, 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:53.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:53.854 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 121600 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:53.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:53.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570147+0000", 2024-09-18T01:09:53.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.85e-07 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570151+0000", 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:53.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570162+0000", 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0317e-05 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570167+0000", 2024-09-18T01:09:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570207+0000", 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0543999999999997e-05 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570262+0000", 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4539000000000001e-05 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:53.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570403+0000", 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140846 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570909+0000", 2024-09-18T01:09:53.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050672900000000003 2024-09-18T01:09:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604784+0000", 2024-09-18T01:09:53.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033874597999999999 2024-09-18T01:09:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604816+0000", 2024-09-18T01:09:53.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1668000000000002e-05 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604841+0000", 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999999e-05 2024-09-18T01:09:53.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:53.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:53.867 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:53.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:53.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:09:53.872 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:09:53.873 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:53.873 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:09:53.874 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:53.867+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:53.969 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:53.967+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:54.017 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:54.017 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:54.017 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:54.017 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:54.025 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:54.049 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:54.066 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:54.084 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:54.084 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:54.100 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:54.100 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:54.110 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:54.108+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:54.117 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:54.117 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:54.133 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:54.134 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:54.150 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:54.150 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:54.166 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:54.167 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:54.171 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:54.169+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:54.171 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.171 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.181 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:54.272 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:54.269+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:54.325 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.325 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.335 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:54.376 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:54.374+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:54.473 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:54.472+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:54.478 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.478 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:09:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:09:54.574 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:54.572+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:54.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:09:54.595 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.595 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:54.675 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:54.673+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:54.747 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.747 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.756 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:54.775 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:54.773+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:54.862 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.862 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:54.915 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:54.913+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:54.974 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:54.974 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:54.977 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:54.975+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:54.983 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:09:55.078 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:55.077+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:55.126 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:55.126 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:55.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:09:55.179 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:55.178+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:55.278 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:55.279 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:55.279 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:55.278+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:55.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:09:55.380 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:55.379+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:55.431 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:55.431 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:55.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:09:55.481 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:55.480+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:55.564 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:09:55.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:09:55.583 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:55.581+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:55.588 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:55.588 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:55.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:09:55.684 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:55.682+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:55.769 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:55.769 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:55.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:09:55.825 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:55.821+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:55.885 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:55.883+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:55.988 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:55.984+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:56.014 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:56.014 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:56.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:09:56.112 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:56.110+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:56.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:09:56.228 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:56.224+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:56.229 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:56.229 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:56.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:09:56.290 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:56.288+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:56.338 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:56.338 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:56.358 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:56.358 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:56.375 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:56.375 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:56.390 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:56.388+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:56.393 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:56.393 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:56.402 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:09:56.491 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:56.489+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:56.504 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:56.504 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:56.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:09:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:09:56.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:09:56.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:09:56.593 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:56.593 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:56.594 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:56.591+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:56.610 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:56.611 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:56.628 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:56.628 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:56.645 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:56.645 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:56.663 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:56.667 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:56.667 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:09:56.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:09:56.667 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:09:56.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036875143000003, 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:09:56.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:09:56.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:09:56.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:56.672 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036850158, 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:09:56.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036830379000001, 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.677 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:09:56.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036803976999998, 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:09:56.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:09:56.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036782481000003, 2024-09-18T01:09:56.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:09:56.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:09:56.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036763184999998, 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.691 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:09:56.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036745351, 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036719288, 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036699933000001, 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:09:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:09:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036669689, 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:09:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:09:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:09:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036651094, 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:09:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:09:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036632535000003, 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036614481000001, 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:09:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:09:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036596721999999, 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:09:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:09:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:09:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036570453000003, 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036551832999997, 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:09:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:09:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.036533583000001, 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:09:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:09:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.035982208, 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:09:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:09:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:09:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.034383222000002, 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:09:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.030617235000001, 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:09:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:09:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:09:56.756 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:56.758 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:09:56.759 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:56.755+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:56.858 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:56.857+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185318 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738882+0000", 2024-09-18T01:09:56.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14150822099999999, 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14152509799999999, 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185318 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738882+0000", 2024-09-18T01:09:56.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738882+0000", 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738882+0000", 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738884+0000", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738891+0000", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.251e-06 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738896+0000", 2024-09-18T01:09:56.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.899e-06 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739258+0000", 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036204100000000001 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739262+0000", 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3889999999999998e-06 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185612 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744479+0000", 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13591081399999999, 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13598882300000001, 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185612 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744479+0000", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744479+0000", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744480+0000", 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7899999999999999e-07 2024-09-18T01:09:56.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744482+0000", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0420000000000001e-06 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744488+0000", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6390000000000003e-06 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744493+0000", 2024-09-18T01:09:56.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744768+0000", 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027496200000000001 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744774+0000", 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7200000000000003e-06 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185906 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753235+0000", 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12715468399999999, 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12726812800000001, 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185906 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753235+0000", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753235+0000", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753236+0000", 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753237+0000", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.540000000000001e-07 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753240+0000", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1250000000000001e-06 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753242+0000", 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:09:56.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753531+0000", 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028878699999999998 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753537+0000", 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2489999999999998e-06 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12659938400000001, 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12675944, 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:56.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:09:56.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.906 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185319 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738910+0000", 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.141480413, 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14167485699999999, 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185319 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738910+0000", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738910+0000", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738910+0000", 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:09:56.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738913+0000", 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.43e-06 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738919+0000", 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2829999999999998e-06 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738924+0000", 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8439999999999997e-06 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739293+0000", 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036956599999999999 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739301+0000", 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2960000000000001e-06 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185613 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744501+0000", 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.135889115, 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13611595100000001, 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185613 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.911 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744501+0000", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744501+0000", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744501+0000", 2024-09-18T01:09:56.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744503+0000", 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9910000000000001e-06 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744510+0000", 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5780000000000004e-06 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744515+0000", 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7049999999999998e-06 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744801+0000", 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028591199999999998 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744807+0000", 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.64e-06 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185907 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753245+0000", 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12714484000000001, 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.127403873, 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185907 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753245+0000", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753245+0000", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753245+0000", 2024-09-18T01:09:56.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.05e-07 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753247+0000", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0240000000000001e-06 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753250+0000", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.022e-06 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753252+0000", 2024-09-18T01:09:56.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.734e-06 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753573+0000", 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000320725 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753580+0000", 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126570611, 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12688327799999999, 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:09:56.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185320 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738930+0000", 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14146039299999999, 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14184032699999999, 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185320 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738930+0000", 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738930+0000", 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738930+0000", 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738932+0000", 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738938+0000", 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738943+0000", 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739325+0000", 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038249699999999998 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739328+0000", 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185614 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744537+0000", 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13585298800000001, 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13629793600000001, 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185614 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744537+0000", 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744537+0000", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744538+0000", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0800000000000004e-07 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744541+0000", 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6210000000000001e-06 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744550+0000", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5429999999999994e-06 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744556+0000", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1569999999999997e-06 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744834+0000", 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027824300000000001 2024-09-18T01:09:56.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744841+0000", 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1229999999999997e-06 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185908 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753255+0000", 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12713512799999999, 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12763118900000001, 2024-09-18T01:09:56.931 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185908 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753255+0000", 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753255+0000", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753255+0000", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753256+0000", 2024-09-18T01:09:56.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.032e-06 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753259+0000", 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0510000000000001e-06 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753262+0000", 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9730000000000002e-06 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753612+0000", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035014900000000001 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753619+0000", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:56.935 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12654291500000001, 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12709469800000001, 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:56.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185321 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738962+0000", 2024-09-18T01:09:56.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14142786399999999, 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.142042694, 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185321 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738962+0000", 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738962+0000", 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738963+0000", 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738965+0000", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738972+0000", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1999999999999997e-06 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738977+0000", 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.967e-06 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739357+0000", 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037985100000000001 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739364+0000", 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9469999999999997e-06 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185615 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744566+0000", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.135823846, 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13648308000000001, 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185615 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744566+0000", 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744566+0000", 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744567+0000", 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744569+0000", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4119999999999999e-06 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744575+0000", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9920000000000004e-06 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744580+0000", 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.605e-06 2024-09-18T01:09:56.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744867+0000", 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028682000000000002 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744873+0000", 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1789999999999996e-06 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185909 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753275+0000", 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12711507699999999, 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12781182799999999, 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185909 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753275+0000", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753275+0000", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753275+0000", 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6199999999999999e-07 2024-09-18T01:09:56.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753277+0000", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4869999999999999e-06 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753280+0000", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.332e-06 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753284+0000", 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-18T01:09:56.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753652+0000", 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000368048 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753658+0000", 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4999999999999996e-06 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.951 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12650846700000001, 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12723753700000001, 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185322 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738985+0000", 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14140530500000001, 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.142167237, 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185322 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738985+0000", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738985+0000", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738985+0000", 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738988+0000", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738994+0000", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1129999999999997e-06 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738998+0000", 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7369999999999997e-06 2024-09-18T01:09:56.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739381+0000", 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000382526 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739384+0000", 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1549999999999999e-06 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.959 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185616 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744706+0000", 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13568411999999999, 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13647831799999999, 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185616 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744706+0000", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744706+0000", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744707+0000", 2024-09-18T01:09:56.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744709+0000", 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8380000000000002e-06 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744716+0000", 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3430000000000001e-06 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744722+0000", 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7120000000000003e-06 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744896+0000", 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017324800000000001 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744908+0000", 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2167000000000001e-05 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185910 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753308+0000", 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.127082151, 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.127908257, 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185910 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753308+0000", 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753308+0000", 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753308+0000", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.91e-07 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753310+0000", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6139999999999999e-06 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753314+0000", 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7740000000000002e-06 2024-09-18T01:09:56.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753316+0000", 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7420000000000002e-06 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753688+0000", 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037207499999999998 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753693+0000", 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.476e-06 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12647862100000001, 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12733672500000001, 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:09:56.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:09:56.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185454 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741033+0000", 2024-09-18T01:09:56.972 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.139357386, 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14024843000000001, 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185454 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741033+0000", 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741033+0000", 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741034+0000", 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3200000000000004e-07 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741038+0000", 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1829999999999996e-06 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741061+0000", 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3414999999999999e-05 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741072+0000", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0472000000000001e-05 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741124+0000", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2096999999999999e-05 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741132+0000", 2024-09-18T01:09:56.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7390000000000005e-06 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185748 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.746310+0000", 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13408052000000001, 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13500390200000001, 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185748 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746310+0000", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746310+0000", 2024-09-18T01:09:56.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746311+0000", 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4e-07 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746315+0000", 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746330+0000", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4568e-05 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746337+0000", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5009999999999998e-06 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746391+0000", 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3773000000000002e-05 2024-09-18T01:09:56.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746399+0000", 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2840000000000006e-06 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185929 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753320+0000", 2024-09-18T01:09:56.981 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.127070447, 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12804194799999999, 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185929 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753320+0000", 2024-09-18T01:09:56.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753320+0000", 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753320+0000", 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753327+0000", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9859999999999997e-06 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753331+0000", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7290000000000002e-06 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753333+0000", 2024-09-18T01:09:56.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8279999999999999e-06 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753725+0000", 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000391853 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753732+0000", 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4529999999999999e-06 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12645643000000001, 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.127458878, 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:09:56.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:56.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:09:56.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185455 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741095+0000", 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13929507599999999, 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140328062, 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185455 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741095+0000", 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741095+0000", 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741096+0000", 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741102+0000", 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.438e-06 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741110+0000", 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0220000000000004e-06 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741116+0000", 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8170000000000004e-06 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741155+0000", 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9110999999999999e-05 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741161+0000", 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4790000000000001e-06 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185749 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.746354+0000", 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.134036398, 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.135099565, 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.994 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185749 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746354+0000", 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746354+0000", 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746354+0000", 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9699999999999996e-07 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:56.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746357+0000", 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9440000000000001e-06 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746363+0000", 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8640000000000001e-06 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746368+0000", 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746425+0000", 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7129000000000001e-05 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746432+0000", 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12641952400000001, 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12751351399999999, 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:56.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:09:57.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:09:57.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185456 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741127+0000", 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13926288100000001, 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140387598, 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185456 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741127+0000", 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741127+0000", 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741128+0000", 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741130+0000", 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.452e-06 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741137+0000", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741142+0000", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741176+0000", 2024-09-18T01:09:57.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4177000000000002e-05 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741179+0000", 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1719999999999999e-06 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185750 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.746409+0000", 2024-09-18T01:09:57.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13398095800000001, 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13515556200000001, 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185750 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746409+0000", 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746409+0000", 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746410+0000", 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5799999999999999e-07 2024-09-18T01:09:57.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746413+0000", 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9079999999999999e-06 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746595+0000", 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018187699999999999 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746601+0000", 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8619999999999997e-06 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746797+0000", 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019595300000000001 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746806+0000", 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6079999999999996e-06 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186060 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753372+0000", 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12701770100000001, 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12822301899999999, 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186060 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753372+0000", 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753372+0000", 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753373+0000", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753374+0000", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.223e-06 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753378+0000", 2024-09-18T01:09:57.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0509999999999998e-06 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753382+0000", 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1269999999999996e-06 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753755+0000", 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037306600000000002 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753758+0000", 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3459999999999998e-06 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12639840899999999, 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12763495699999999, 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185457 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741151+0000", 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13923901599999999, 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14050621699999999, 2024-09-18T01:09:57.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185457 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741151+0000", 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741151+0000", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741152+0000", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741154+0000", 2024-09-18T01:09:57.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7030000000000002e-06 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741161+0000", 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3210000000000001e-06 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741170+0000", 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5789999999999993e-06 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741204+0000", 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3550999999999998e-05 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741207+0000", 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1420000000000002e-06 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185751 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.746619+0000", 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13377135800000001, 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13506943699999999, 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185751 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746619+0000", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746619+0000", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746619+0000", 2024-09-18T01:09:57.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.44e-07 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746623+0000", 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.517e-06 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746632+0000", 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6840000000000002e-06 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.746643+0000", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0910999999999999e-05 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747085+0000", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044207099999999999 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747093+0000", 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1470000000000001e-06 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.028 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12636788400000001, 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12769820300000001, 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:09:57.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185458 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741186+0000", 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13920458199999999, 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140565421, 2024-09-18T01:09:57.033 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185458 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741186+0000", 2024-09-18T01:09:57.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741186+0000", 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741186+0000", 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000005e-07 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741196+0000", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9899999999999992e-06 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741203+0000", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0809999999999996e-06 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741209+0000", 2024-09-18T01:09:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741238+0000", 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9079999999999999e-05 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741241+0000", 2024-09-18T01:09:57.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.286e-06 2024-09-18T01:09:57.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185752 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747026+0000", 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13336382399999999, 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.134755402, 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185752 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747026+0000", 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747026+0000", 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747027+0000", 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4799999999999998e-07 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747040+0000", 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3074999999999999e-05 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747059+0000", 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9436000000000002e-05 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747067+0000", 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0029999999999994e-06 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747122+0000", 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.435e-05 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747130+0000", 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186078 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753422+0000", 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12696772200000001, 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.128389735, 2024-09-18T01:09:57.045 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186078 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753422+0000", 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753422+0000", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753423+0000", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9999999999999998e-07 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753425+0000", 2024-09-18T01:09:57.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1050000000000002e-06 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753431+0000", 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9479999999999996e-06 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753435+0000", 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0849999999999999e-06 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035141399999999998 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.049 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186373 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758521+0000", 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.121869594, 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12332240899999999, 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186373 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758521+0000", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758521+0000", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758522+0000", 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.95e-06 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758530+0000", 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758549+0000", 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8961e-05 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758558+0000", 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5760000000000006e-06 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758628+0000", 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0412000000000003e-05 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758637+0000", 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.098e-06 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.053 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185459 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741218+0000", 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13917189299999999, 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14065545099999999, 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.054 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185459 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741218+0000", 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741218+0000", 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741219+0000", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6400000000000003e-07 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741225+0000", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.562e-06 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741231+0000", 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1940000000000003e-06 2024-09-18T01:09:57.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741236+0000", 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741260+0000", 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3788e-05 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741263+0000", 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1640000000000001e-06 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185753 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747079+0000", 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13331072199999999, 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13484900399999999, 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185753 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747079+0000", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747079+0000", 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747080+0000", 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.37e-07 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747084+0000", 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3390000000000003e-06 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747091+0000", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0790000000000003e-06 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747096+0000", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.921e-06 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747154+0000", 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7241000000000003e-05 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747160+0000", 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3430000000000001e-06 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186079 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753444+0000", 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12694632, 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12851711099999999, 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186079 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753444+0000", 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753444+0000", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753444+0000", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.91e-07 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753446+0000", 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9149999999999999e-06 2024-09-18T01:09:57.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753450+0000", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6849999999999999e-06 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753453+0000", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753821+0000", 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036823599999999998 2024-09-18T01:09:57.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753824+0000", 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.298e-06 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186374 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758579+0000", 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.121811318, 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.123414178, 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186374 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758579+0000", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758579+0000", 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758579+0000", 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4300000000000003e-07 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758582+0000", 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6690000000000002e-06 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759241+0000", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00065908700000000004 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759262+0000", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.092e-05 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759326+0000", 2024-09-18T01:09:57.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880999999999994e-05 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759335+0000", 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2879999999999998e-06 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185460 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741245+0000", 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13914553299999999, 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14078057999999999, 2024-09-18T01:09:57.070 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185460 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741245+0000", 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741245+0000", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741245+0000", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9799999999999999e-07 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741247+0000", 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.939e-06 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741253+0000", 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741258+0000", 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741282+0000", 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3955999999999999e-05 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741285+0000", 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1350000000000001e-06 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185754 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747126+0000", 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.133264146, 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13493111499999999, 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185754 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747126+0000", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747126+0000", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747127+0000", 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6000000000000003e-07 2024-09-18T01:09:57.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747131+0000", 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0210000000000001e-06 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747140+0000", 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9970000000000004e-06 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747146+0000", 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747182+0000", 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5719999999999997e-05 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747188+0000", 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2249999999999997e-06 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186080 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753457+0000", 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12693347299999999, 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12863246, 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186080 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753457+0000", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753457+0000", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753457+0000", 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:09:57.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753458+0000", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.04e-06 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753462+0000", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.985e-06 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753466+0000", 2024-09-18T01:09:57.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2830000000000003e-06 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753854+0000", 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038787499999999998 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753863+0000", 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6540000000000005e-06 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186375 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759284+0000", 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.121106382, 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12283754299999999, 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186375 2024-09-18T01:09:57.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759284+0000", 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759284+0000", 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759285+0000", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8299999999999996e-07 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759292+0000", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3479999999999998e-06 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759302+0000", 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7110000000000007e-06 2024-09-18T01:09:57.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759308+0000", 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759367+0000", 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9182999999999999e-05 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759374+0000", 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6499999999999999e-06 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185461 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741266+0000", 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13912442899999999, 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140887756, 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.087 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185461 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741266+0000", 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741266+0000", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741266+0000", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741268+0000", 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741274+0000", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1680000000000001e-06 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741279+0000", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6859999999999997e-06 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741306+0000", 2024-09-18T01:09:57.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6554999999999999e-05 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741311+0000", 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-06 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185755 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747158+0000", 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13323257799999999, 2024-09-18T01:09:57.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13502757100000001, 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185755 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747158+0000", 2024-09-18T01:09:57.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747158+0000", 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747158+0000", 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1500000000000005e-07 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747160+0000", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3190000000000002e-06 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747166+0000", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8769999999999998e-06 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747171+0000", 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747213+0000", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2029e-05 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747219+0000", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1709999999999999e-06 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186081 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753472+0000", 2024-09-18T01:09:57.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12691839199999999, 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.128745462, 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186081 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753472+0000", 2024-09-18T01:09:57.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753472+0000", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753472+0000", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2099999999999998e-07 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753474+0000", 2024-09-18T01:09:57.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.779e-06 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753480+0000", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9030000000000001e-06 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753485+0000", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753931+0000", 2024-09-18T01:09:57.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044665200000000002 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7621999999999998e-05 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186376 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759320+0000", 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.121070148, 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.122929356, 2024-09-18T01:09:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186376 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759320+0000", 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759320+0000", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759321+0000", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6100000000000001e-07 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759323+0000", 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6680000000000001e-06 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759330+0000", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0620000000000003e-06 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759336+0000", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3499999999999996e-06 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759554+0000", 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021887100000000001 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759563+0000", 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185462 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741287+0000", 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13910304500000001, 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140995017, 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185462 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741287+0000", 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741287+0000", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741288+0000", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741290+0000", 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.119e-06 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741297+0000", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741302+0000", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741329+0000", 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7287000000000001e-05 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741335+0000", 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.429e-06 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185756 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747178+0000", 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.133212197, 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13513656399999999, 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185756 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747178+0000", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747178+0000", 2024-09-18T01:09:57.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747178+0000", 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747180+0000", 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747186+0000", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7270000000000004e-06 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747191+0000", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747387+0000", 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019509099999999999 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747396+0000", 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9609999999999994e-06 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186082 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753490+0000", 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126900079, 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12885690499999999, 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.111 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186082 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753490+0000", 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753490+0000", 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753490+0000", 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753492+0000", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753499+0000", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753503+0000", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0369999999999996e-06 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753989+0000", 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048642800000000001 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753996+0000", 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8669999999999996e-06 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186377 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759394+0000", 2024-09-18T01:09:57.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120996478, 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.122985762, 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186377 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759394+0000", 2024-09-18T01:09:57.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759394+0000", 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759394+0000", 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9800000000000003e-07 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759408+0000", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759417+0000", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0950000000000002e-06 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759424+0000", 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6039999999999998e-06 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759598+0000", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000174294 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759604+0000", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0390000000000001e-06 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185463 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741310+0000", 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.139080064, 2024-09-18T01:09:57.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.141102267, 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185463 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741310+0000", 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741310+0000", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741311+0000", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7300000000000001e-07 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741313+0000", 2024-09-18T01:09:57.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741319+0000", 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8109999999999999e-06 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741323+0000", 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741349+0000", 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5910000000000001e-05 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741352+0000", 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2710000000000001e-06 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185757 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747249+0000", 2024-09-18T01:09:57.123 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13314145899999999, 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13519645399999999, 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185757 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747249+0000", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747249+0000", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747249+0000", 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1399999999999996e-07 2024-09-18T01:09:57.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747253+0000", 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.985e-06 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747263+0000", 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2590000000000006e-06 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747287+0000", 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4810000000000001e-05 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747420+0000", 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013287999999999999 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747427+0000", 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6409999999999996e-06 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186083 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753524+0000", 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126866114, 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.128982657, 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186083 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753524+0000", 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753524+0000", 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753524+0000", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6400000000000003e-07 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753526+0000", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7069999999999999e-06 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753530+0000", 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2370000000000003e-06 2024-09-18T01:09:57.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753533+0000", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9500000000000001e-06 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754027+0000", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049329500000000004 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754034+0000", 2024-09-18T01:09:57.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0700000000000001e-06 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186378 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759436+0000", 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12095439299999999, 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12310415199999999, 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186378 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759436+0000", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759436+0000", 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759436+0000", 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759439+0000", 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.447e-06 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759446+0000", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1019999999999999e-06 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759451+0000", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.22e-06 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000186412 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759644+0000", 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0839999999999997e-06 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185464 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741331+0000", 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.139059285, 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14124182399999999, 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.136 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185464 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741331+0000", 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741331+0000", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741331+0000", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741333+0000", 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741340+0000", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1480000000000003e-06 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741344+0000", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741373+0000", 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8890999999999998e-05 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741379+0000", 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185758 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747439+0000", 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13295111100000001, 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13516663100000001, 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185758 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747439+0000", 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747439+0000", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747440+0000", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2200000000000003e-07 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747444+0000", 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7950000000000001e-06 2024-09-18T01:09:57.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747456+0000", 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2133999999999999e-05 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747464+0000", 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2120000000000003e-06 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747508+0000", 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4342999999999999e-05 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747515+0000", 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186084 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753537+0000", 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12685314, 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12910174799999999, 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186084 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753537+0000", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753537+0000", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753537+0000", 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.35e-07 2024-09-18T01:09:57.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753538+0000", 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0419999999999999e-06 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753543+0000", 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.758e-06 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753547+0000", 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999996e-06 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754082+0000", 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053474399999999997 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754091+0000", 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0550000000000005e-06 2024-09-18T01:09:57.148 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186379 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759459+0000", 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120930841, 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12321208, 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186379 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759459+0000", 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759459+0000", 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759460+0000", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759462+0000", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3779999999999999e-06 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759469+0000", 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5259999999999999e-06 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759474+0000", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759669+0000", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019552099999999999 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759675+0000", 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0510000000000002e-06 2024-09-18T01:09:57.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185465 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741353+0000", 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13903722099999999, 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14135144199999999, 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185465 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741353+0000", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741353+0000", 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741353+0000", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741356+0000", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741361+0000", 2024-09-18T01:09:57.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7389999999999996e-06 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741366+0000", 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7929999999999997e-06 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741392+0000", 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6177999999999999e-05 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741398+0000", 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7080000000000002e-06 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185759 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747479+0000", 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13291118399999999, 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.135257668, 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.157 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185759 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747479+0000", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747479+0000", 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747479+0000", 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747482+0000", 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747489+0000", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.669e-06 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747494+0000", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1919999999999996e-06 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747543+0000", 2024-09-18T01:09:57.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.871e-05 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747550+0000", 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7789999999999996e-06 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186085 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753566+0000", 2024-09-18T01:09:57.161 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12682461, 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.129203549, 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186085 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753566+0000", 2024-09-18T01:09:57.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753566+0000", 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753566+0000", 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753567+0000", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.618e-06 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753571+0000", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9999999999999998e-06 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753574+0000", 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6429999999999999e-06 2024-09-18T01:09:57.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754127+0000", 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055310699999999995 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754134+0000", 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186380 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759481+0000", 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120909081, 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12332169, 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186380 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.166 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759481+0000", 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759481+0000", 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759481+0000", 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759484+0000", 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759490+0000", 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.246e-06 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759496+0000", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2989999999999996e-06 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759705+0000", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000209445 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759711+0000", 2024-09-18T01:09:57.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0870000000000003e-06 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185466 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741374+0000", 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13901656700000001, 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14146202799999999, 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185466 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741374+0000", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741374+0000", 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741374+0000", 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741376+0000", 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741382+0000", 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741387+0000", 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6539999999999998e-06 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741417+0000", 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0295999999999999e-05 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741423+0000", 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5439999999999998e-06 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.174 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185760 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747502+0000", 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.132888161, 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13536626399999999, 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185760 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747502+0000", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747502+0000", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747502+0000", 2024-09-18T01:09:57.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747505+0000", 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747512+0000", 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.148e-06 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747517+0000", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9849999999999997e-06 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747572+0000", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4938000000000002e-05 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747578+0000", 2024-09-18T01:09:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186086 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753580+0000", 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126809914, 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12932951100000001, 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186086 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753580+0000", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753580+0000", 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753580+0000", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753582+0000", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.285e-06 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753586+0000", 2024-09-18T01:09:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753589+0000", 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2150000000000001e-06 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754180+0000", 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00059036799999999997 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754200+0000", 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0621000000000001e-05 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12088763, 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12344043, 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:09:57.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185467 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741394+0000", 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13899642100000001, 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14158205400000001, 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185467 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741394+0000", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741394+0000", 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741394+0000", 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741396+0000", 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741402+0000", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2600000000000002e-06 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741407+0000", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9280000000000001e-06 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741447+0000", 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9523000000000001e-05 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741453+0000", 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.913e-06 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185761 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747554+0000", 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.132836342, 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13545468499999999, 2024-09-18T01:09:57.191 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185761 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747554+0000", 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747554+0000", 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747554+0000", 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747558+0000", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747567+0000", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2180000000000005e-06 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747573+0000", 2024-09-18T01:09:57.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1009999999999997e-06 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747608+0000", 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4739000000000001e-05 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747614+0000", 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186087 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753593+0000", 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126797033, 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12944805600000001, 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186087 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753593+0000", 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753593+0000", 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753593+0000", 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.11e-07 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753594+0000", 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1119999999999999e-06 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753597+0000", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753601+0000", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.264e-06 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754236+0000", 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00063514400000000003 2024-09-18T01:09:57.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754242+0000", 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.139e-06 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12083377100000001, 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12351754299999999, 2024-09-18T01:09:57.200 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:09:57.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185468 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741415+0000", 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.138975036, 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14169806700000001, 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185468 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741415+0000", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741415+0000", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741415+0000", 2024-09-18T01:09:57.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741417+0000", 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741424+0000", 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741428+0000", 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-06 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741466+0000", 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741472+0000", 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5990000000000001e-06 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185762 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747590+0000", 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13280017299999999, 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.135556276, 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.209 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185762 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747590+0000", 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747590+0000", 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747590+0000", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747593+0000", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747600+0000", 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8040000000000004e-06 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747605+0000", 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1189999999999996e-06 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747639+0000", 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4251000000000001e-05 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747646+0000", 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3889999999999999e-06 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186088 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753604+0000", 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126786382, 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.129589284, 2024-09-18T01:09:57.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186088 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753604+0000", 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753604+0000", 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753604+0000", 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753605+0000", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0899999999999999e-06 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753608+0000", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9799999999999998e-06 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753610+0000", 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4210000000000002e-06 2024-09-18T01:09:57.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754278+0000", 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066747599999999996 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754286+0000", 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120756349, 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.123623618, 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185469 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.741445+0000", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.138945398, 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14187704200000001, 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185469 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741445+0000", 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741445+0000", 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741445+0000", 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7100000000000002e-07 2024-09-18T01:09:57.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741447+0000", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0789999999999999e-06 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741454+0000", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3849999999999999e-06 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741458+0000", 2024-09-18T01:09:57.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.425e-06 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741492+0000", 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3519000000000002e-05 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.741495+0000", 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185763 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.747616+0000", 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13277456300000001, 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.135766476, 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185763 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747616+0000", 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747616+0000", 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747616+0000", 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747620+0000", 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8569999999999997e-06 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747627+0000", 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3529999999999997e-06 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747632+0000", 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:09:57.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747676+0000", 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3479999999999997e-05 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.747685+0000", 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.577e-06 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.230 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186089 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753614+0000", 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126776375, 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.129802526, 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186089 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753614+0000", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753614+0000", 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753614+0000", 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0699999999999999e-07 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753615+0000", 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753618+0000", 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753621+0000", 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5000000000000002e-06 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754319+0000", 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069890299999999996 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754326+0000", 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3289999999999999e-06 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120726235, 2024-09-18T01:09:57.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12378463200000001, 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:09:57.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185307 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738461+0000", 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14192918800000001, 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14502229, 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185307 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738461+0000", 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738461+0000", 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738463+0000", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.52e-06 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738472+0000", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3750000000000003e-06 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738493+0000", 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0696999999999999e-05 2024-09-18T01:09:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738503+0000", 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0518e-05 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738581+0000", 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8021000000000004e-05 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738588+0000", 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2529999999999998e-06 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185601 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744044+0000", 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13634605499999999, 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13947145399999999, 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185601 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744044+0000", 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744044+0000", 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744045+0000", 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.176e-06 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744051+0000", 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744066+0000", 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5007e-05 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744074+0000", 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6750000000000002e-06 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744340+0000", 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026671300000000002 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744348+0000", 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8730000000000007e-06 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185895 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.752971+0000", 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12741954899999999, 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13057698200000001, 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185895 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752971+0000", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752971+0000", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752972+0000", 2024-09-18T01:09:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.435e-06 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752978+0000", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3879999999999997e-06 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752992+0000", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3772e-05 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.752999+0000", 2024-09-18T01:09:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4400000000000002e-06 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753057+0000", 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8187999999999999e-05 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753066+0000", 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.431e-06 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186090 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753623+0000", 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12676673399999999, 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12995627600000001, 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186090 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753623+0000", 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753623+0000", 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753624+0000", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753625+0000", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.838e-06 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753630+0000", 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0599999999999998e-06 2024-09-18T01:09:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753634+0000", 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8909999999999997e-06 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754361+0000", 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00072667000000000001 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754368+0000", 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4459999999999998e-06 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12067631700000001, 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.123897506, 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:09:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185308 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738527+0000", 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14186310499999999, 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14511666500000001, 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185308 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738527+0000", 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738527+0000", 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738528+0000", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.99e-07 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738531+0000", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738542+0000", 2024-09-18T01:09:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1232e-05 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738549+0000", 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3130000000000004e-06 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738623+0000", 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4877999999999997e-05 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738632+0000", 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3070000000000003e-06 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185602 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744101+0000", 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13628958899999999, 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.139575328, 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185602 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744101+0000", 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744101+0000", 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744101+0000", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3200000000000005e-07 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744104+0000", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5069999999999999e-06 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744113+0000", 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2690000000000005e-06 2024-09-18T01:09:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744119+0000", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3679999999999998e-06 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744390+0000", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000270795 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744397+0000", 2024-09-18T01:09:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8410000000000002e-06 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185896 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753013+0000", 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.127377353, 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.130695123, 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185896 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753013+0000", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753013+0000", 2024-09-18T01:09:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753013+0000", 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4200000000000002e-07 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753015+0000", 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6810000000000001e-06 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753020+0000", 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8790000000000002e-06 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753024+0000", 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8890000000000004e-06 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753114+0000", 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0364000000000004e-05 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753124+0000", 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0223e-05 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186091 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753646+0000", 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12674406499999999, 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13009356699999999, 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186091 2024-09-18T01:09:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753646+0000", 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753646+0000", 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753647+0000", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8400000000000004e-07 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753649+0000", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753654+0000", 2024-09-18T01:09:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6630000000000001e-06 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753656+0000", 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.74e-06 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754402+0000", 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074524900000000002 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754408+0000", 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0530000000000003e-06 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12064273, 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.124039131, 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185309 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738563+0000", 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.141826855, 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14525366200000001, 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185309 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738563+0000", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738563+0000", 2024-09-18T01:09:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738564+0000", 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8599999999999998e-07 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738567+0000", 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.824e-06 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738578+0000", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.112e-05 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738583+0000", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4299999999999997e-06 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738673+0000", 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9819000000000002e-05 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738684+0000", 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1127e-05 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185603 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744129+0000", 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13626102900000001, 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.139718272, 2024-09-18T01:09:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185603 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744129+0000", 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744129+0000", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744130+0000", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1900000000000003e-07 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744132+0000", 2024-09-18T01:09:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744137+0000", 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5079999999999996e-06 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744140+0000", 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.253e-06 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744451+0000", 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031023299999999998 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744459+0000", 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7399999999999993e-06 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185897 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753043+0000", 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12734731199999999, 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.130834598, 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185897 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753043+0000", 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753043+0000", 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753043+0000", 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1699999999999999e-07 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753045+0000", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.885e-06 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753049+0000", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0659999999999997e-06 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753052+0000", 2024-09-18T01:09:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.208e-06 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753162+0000", 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109574 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753169+0000", 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7290000000000003e-06 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186092 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753661+0000", 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126729388, 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13025292899999999, 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186092 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753661+0000", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753661+0000", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753661+0000", 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1500000000000001e-07 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753662+0000", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.11e-06 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753665+0000", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753668+0000", 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754442+0000", 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00077389799999999997 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754449+0000", 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1269999999999997e-06 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120618196, 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.124171956, 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185310 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738591+0000", 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14179941800000001, 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14538367799999999, 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185310 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738591+0000", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738591+0000", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738591+0000", 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738594+0000", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4619999999999999e-06 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738599+0000", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8549999999999998e-06 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738605+0000", 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738723+0000", 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118554 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738730+0000", 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2609999999999996e-06 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185604 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744147+0000", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13624360899999999, 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13985834999999999, 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185604 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744147+0000", 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744147+0000", 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744147+0000", 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744149+0000", 2024-09-18T01:09:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.407e-06 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744157+0000", 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7570000000000004e-06 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744162+0000", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.865e-06 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744499+0000", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033704999999999998 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744506+0000", 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.798e-06 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185898 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753057+0000", 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12733361100000001, 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13097845, 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185898 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753057+0000", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753057+0000", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753057+0000", 2024-09-18T01:09:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4699999999999998e-07 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753058+0000", 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1680000000000001e-06 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753061+0000", 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4750000000000002e-06 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753066+0000", 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2239999999999997e-06 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753206+0000", 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013989699999999999 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753210+0000", 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8470000000000003e-06 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186093 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753671+0000", 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12671892700000001, 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13039440099999999, 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186093 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753671+0000", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753671+0000", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753671+0000", 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753672+0000", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0669999999999999e-06 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753676+0000", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1190000000000001e-06 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753679+0000", 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754483+0000", 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00080409899999999996 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754489+0000", 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2339999999999999e-06 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.120432914, 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.12413914500000001, 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:09:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185311 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738611+0000", 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14177904, 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.145515752, 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185311 2024-09-18T01:09:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738611+0000", 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738611+0000", 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738611+0000", 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738614+0000", 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.204e-06 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738619+0000", 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7289999999999997e-06 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738628+0000", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0189999999999995e-06 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738759+0000", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013009999999999999 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738765+0000", 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:09:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185605 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744169+0000", 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.136221011, 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13998792600000001, 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185605 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744169+0000", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744169+0000", 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744170+0000", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744181+0000", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1062e-05 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744195+0000", 2024-09-18T01:09:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4589999999999999e-05 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744204+0000", 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2669999999999995e-06 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744539+0000", 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033428 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744545+0000", 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.331e-06 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185899 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753071+0000", 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12731873399999999, 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.131115487, 2024-09-18T01:09:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185899 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753071+0000", 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753071+0000", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753072+0000", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753083+0000", 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1539000000000001e-05 2024-09-18T01:09:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753090+0000", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8809999999999998e-06 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753102+0000", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1447000000000001e-05 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753244+0000", 2024-09-18T01:09:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014271999999999999 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753251+0000", 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8109999999999997e-06 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186225 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753682+0000", 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12670859900000001, 2024-09-18T01:09:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13053537000000001, 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186225 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753682+0000", 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753682+0000", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753682+0000", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.05e-07 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753683+0000", 2024-09-18T01:09:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02e-06 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753686+0000", 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753691+0000", 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6330000000000001e-06 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754523+0000", 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083148399999999996 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754529+0000", 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9150000000000001e-06 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185312 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738635+0000", 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14175525999999999, 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14561221899999999, 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185312 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738635+0000", 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738635+0000", 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738635+0000", 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738637+0000", 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2290000000000002e-06 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738643+0000", 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4620000000000004e-06 2024-09-18T01:09:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738648+0000", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8919999999999999e-06 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738919+0000", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027166000000000001 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738927+0000", 2024-09-18T01:09:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8369999999999997e-06 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185606 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744240+0000", 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13615059299999999, 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14003796499999999, 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185606 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744240+0000", 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744240+0000", 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744240+0000", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.61e-07 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744245+0000", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744273+0000", 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7798e-05 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744281+0000", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4409999999999993e-06 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744571+0000", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000289952 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744577+0000", 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:09:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185900 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753116+0000", 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12727407299999999, 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13119138599999999, 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185900 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753116+0000", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753116+0000", 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753117+0000", 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753119+0000", 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6850000000000001e-06 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753124+0000", 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.843e-06 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753127+0000", 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1420000000000002e-06 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753277+0000", 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015013599999999999 2024-09-18T01:09:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753282+0000", 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5460000000000002e-06 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186226 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753695+0000", 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12669555599999999, 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13064313399999999, 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186226 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753695+0000", 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753695+0000", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753695+0000", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1400000000000001e-07 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753696+0000", 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.116e-06 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753700+0000", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9319999999999998e-06 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753702+0000", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754558+0000", 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00085601899999999996 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754564+0000", 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1979999999999998e-06 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185313 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738674+0000", 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14171642100000001, 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14569423400000001, 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185313 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738674+0000", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738674+0000", 2024-09-18T01:09:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738675+0000", 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9999999999999996e-07 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738677+0000", 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9340000000000002e-06 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738685+0000", 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2439999999999996e-06 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738690+0000", 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2429999999999996e-06 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738971+0000", 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000281358 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738979+0000", 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185607 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744313+0000", 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13607683500000001, 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140108132, 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185607 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744313+0000", 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744313+0000", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744314+0000", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744319+0000", 2024-09-18T01:09:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.498e-06 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744329+0000", 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0497e-05 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744335+0000", 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744608+0000", 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027243999999999999 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744614+0000", 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185901 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753154+0000", 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12723658600000001, 2024-09-18T01:09:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13129890499999999, 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185901 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753154+0000", 2024-09-18T01:09:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753154+0000", 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753154+0000", 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6399999999999998e-07 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753155+0000", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6440000000000001e-06 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753160+0000", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1860000000000002e-06 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753162+0000", 2024-09-18T01:09:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7580000000000001e-06 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753315+0000", 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015263399999999999 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753321+0000", 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2140000000000001e-06 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186227 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753717+0000", 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126673166, 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13076595499999999, 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186227 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753717+0000", 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753717+0000", 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753717+0000", 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753719+0000", 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.486e-06 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753725+0000", 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750000000000004e-06 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753728+0000", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9270000000000001e-06 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754595+0000", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00086762700000000005 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754603+0000", 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.52e-06 2024-09-18T01:09:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185314 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738723+0000", 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14166714, 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14579091699999999, 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185314 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738723+0000", 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738723+0000", 2024-09-18T01:09:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738724+0000", 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4300000000000002e-07 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738736+0000", 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1779e-05 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738749+0000", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3434999999999999e-05 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738757+0000", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2930000000000001e-06 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739022+0000", 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026445499999999997 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739030+0000", 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185608 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744347+0000", 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13604333699999999, 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14019771, 2024-09-18T01:09:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185608 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744347+0000", 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744347+0000", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744347+0000", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2600000000000002e-07 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744350+0000", 2024-09-18T01:09:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5770000000000001e-06 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744356+0000", 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9009999999999999e-06 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744361+0000", 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744640+0000", 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027866099999999998 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744646+0000", 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7970000000000003e-06 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185902 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753166+0000", 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12722441400000001, 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13140901899999999, 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185902 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753166+0000", 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753166+0000", 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753166+0000", 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-07 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753167+0000", 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2049999999999999e-06 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753170+0000", 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2729999999999998e-06 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753173+0000", 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.52e-06 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753357+0000", 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183816 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753363+0000", 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5590000000000002e-06 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186228 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753731+0000", 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12665921799999999, 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.130874091, 2024-09-18T01:09:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186228 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753731+0000", 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753731+0000", 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753731+0000", 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753736+0000", 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.04e-06 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753740+0000", 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.675e-06 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753742+0000", 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5519999999999999e-06 2024-09-18T01:09:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754635+0000", 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089213200000000002 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754642+0000", 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185315 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738771+0000", 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14161900799999999, 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14586439500000001, 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185315 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738771+0000", 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738771+0000", 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738772+0000", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2099999999999996e-07 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738776+0000", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8079999999999998e-06 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738785+0000", 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9800000000000004e-06 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738790+0000", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739070+0000", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027989999999999997 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739104+0000", 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4139e-05 2024-09-18T01:09:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185609 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744385+0000", 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13600551599999999, 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14028112000000001, 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185609 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744385+0000", 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744385+0000", 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744385+0000", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-07 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744391+0000", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999998e-06 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744397+0000", 2024-09-18T01:09:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744402+0000", 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6779999999999999e-06 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744675+0000", 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027269000000000002 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744681+0000", 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2450000000000003e-06 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185903 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753198+0000", 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12719198300000001, 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.131497643, 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185903 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753198+0000", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753198+0000", 2024-09-18T01:09:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753198+0000", 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4299999999999999e-07 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753203+0000", 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3549999999999998e-06 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753206+0000", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4860000000000002e-06 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753209+0000", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6309999999999999e-06 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753406+0000", 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019731499999999999 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753415+0000", 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6330000000000002e-06 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186229 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753753+0000", 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.126636954, 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.130972856, 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186229 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753753+0000", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753753+0000", 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753753+0000", 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753758+0000", 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753761+0000", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299999999999999e-06 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753764+0000", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6060000000000001e-06 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754700+0000", 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00093623399999999998 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754708+0000", 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6699999999999994e-06 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185316 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738832+0000", 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.141558085, 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14592451400000001, 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.410 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185316 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738832+0000", 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738832+0000", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738833+0000", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738836+0000", 2024-09-18T01:09:57.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6059999999999999e-06 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738845+0000", 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3129999999999999e-06 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738851+0000", 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739165+0000", 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031347800000000001 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739173+0000", 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6270000000000006e-06 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185610 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744431+0000", 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.135958783, 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.140355487, 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185610 2024-09-18T01:09:57.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744431+0000", 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744431+0000", 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744432+0000", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744434+0000", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9520000000000001e-06 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744440+0000", 2024-09-18T01:09:57.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0140000000000004e-06 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744443+0000", 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9869999999999999e-06 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744705+0000", 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026234000000000001 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744712+0000", 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.063e-06 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185904 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753212+0000", 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.127177965, 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13160458999999999, 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.419 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185904 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753212+0000", 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753212+0000", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753212+0000", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753213+0000", 2024-09-18T01:09:57.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0920000000000001e-06 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753217+0000", 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0599999999999999e-06 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753219+0000", 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4590000000000001e-06 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753452+0000", 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000232885 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753459+0000", 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8079999999999999e-06 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12662298799999999, 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13107972500000001, 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:09:57.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185317 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.738861+0000", 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14152911300000001, 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14601628799999999, 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185317 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738861+0000", 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738861+0000", 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738861+0000", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3700000000000001e-07 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738864+0000", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4559999999999999e-06 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738870+0000", 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.421e-06 2024-09-18T01:09:57.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.738875+0000", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5390000000000001e-06 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739220+0000", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034528199999999998 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.739227+0000", 2024-09-18T01:09:57.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9709999999999998e-06 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185611 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.744447+0000", 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.13594278300000001, 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.14046387099999999, 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185611 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744447+0000", 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744447+0000", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744448+0000", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744449+0000", 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.773e-06 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744455+0000", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6180000000000003e-06 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744460+0000", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6380000000000003e-06 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744737+0000", 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000277364 2024-09-18T01:09:57.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.744743+0000", 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0090000000000004e-06 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:185905 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753224+0000", 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12716654799999999, 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13171797700000001, 2024-09-18T01:09:57.436 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185905 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753224+0000", 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753224+0000", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753224+0000", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2000000000000001e-07 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753227+0000", 2024-09-18T01:09:57.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7939999999999998e-06 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753230+0000", 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2380000000000002e-06 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753232+0000", 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4770000000000002e-06 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753491+0000", 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025875600000000001 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753498+0000", 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9929999999999998e-06 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3.308dae49 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.440 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.12661341000000001, 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.13119507, 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:09:57.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:09:57.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 130 2024-09-18T01:09:57.445 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:57.447 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:09:57.447 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:57.445+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:57.547 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:57.545+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:57.571 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:09:57.574 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:09:57.597 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:09:57.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:09:57.603 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:57.604 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:57.604 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:57.604 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:57.604 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:57.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:09:57.683 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:57.681+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:57.724 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.999494368, 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:09:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:09:57.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:09:57.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96846784600000002, 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:09:57.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.737 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96845826800000001, 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:09:57.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:09:57.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96844424200000001, 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:09:57.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:09:57.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96841546899999997, 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.748 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:09:57.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:57.753 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96838777300000001, 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:09:57.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96835332500000004, 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:09:57.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:09:57.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:09:57.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96832347900000004, 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:57.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96830128800000004, 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.770 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96826438199999998, 2024-09-18T01:09:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:09:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:09:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96824326699999996, 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:09:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96821274199999996, 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:09:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:09:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.962732488, 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:09:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:09:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:09:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96267862900000001, 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:09:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:09:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:09:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96260120699999996, 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:09:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96257109299999999, 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96252117500000001, 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:09:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96248758800000001, 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:09:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:09:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:09:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96246305399999998, 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:09:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.96227777199999998, 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:09:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:57.833 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:57.835 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:09:57.835 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:57.833+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:57.920 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:57.920 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:09:57.932 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-18T01:09:57.571213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1238068295,"num_objects":1300,"num_object_clones":0,"num_object_copies":2600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1300,"num_whiteouts":0,"num_read":175669,"num_read_kb":1011377,"num_write":22644,"num_write_kb":705582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":766,"num_bytes_recovered":112636048,"num_keys_recovered":99,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15472,"ondisk_log_size":15472,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1053368,"kb_used_data":646872,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":753921352,"statfs":{"total":773094113280,"available":772015464448,"internally_reserved":0,"allocated":662396928,"data_stored":650568616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":410242608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2485753,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":266,"num_read_kb":4258,"num_write":195,"num_write_kb":4926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.782525"},"pg_stats":[{"pgid":"3.1f","version":"158'359","reported_seq":3763,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.987343+0000","last_change":"2024-09-18T01:09:50.886020+0000","last_active":"2024-09-18T01:09:51.987343+0000","last_peered":"2024-09-18T01:09:51.987343+0000","last_clean":"2024-09-18T01:09:51.987343+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:09:51.987343+0000","last_undegraded":"2024-09-18T01:09:51.987343+0000","last_fullsized":"2024-09-18T01:09:51.987343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:59:33.823603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010689580000000001,"stat_sum":{"num_bytes":41730540,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3495,"num_read_kb":30759,"num_write":398,"num_write_kb":27187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1e","version":"158'434","reported_seq":2130,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564078+0000","last_change":"2024-09-18T01:09:50.768682+0000","last_active":"2024-09-18T01:09:54.564078+0000","last_peered":"2024-09-18T01:09:54.564078+0000","last_clean":"2024-09-18T01:09:54.564078+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:09:54.564078+0000","last_undegraded":"2024-09-18T01:09:54.564078+0000","last_fullsized":"2024-09-18T01:09:54.564078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:50.620438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072675599999999995,"stat_sum":{"num_bytes":19075625,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2207,"num_read_kb":25460,"num_write":686,"num_write_kb":19903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1d","version":"157'324","reported_seq":2213,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771331+0000","last_change":"2024-09-18T01:09:50.765539+0000","last_active":"2024-09-18T01:09:51.771331+0000","last_peered":"2024-09-18T01:09:51.771331+0000","last_clean":"2024-09-18T01:09:51.771331+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:09:51.771331+0000","last_undegraded":"2024-09-18T01:09:51.771331+0000","last_fullsized":"2024-09-18T01:09:51.771331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:40.822510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031285699999999999,"stat_sum":{"num_bytes":34477206,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1629,"num_read_kb":32145,"num_write":466,"num_write_kb":36459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1c","version":"158'419","reported_seq":4301,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771446+0000","last_change":"2024-09-18T01:09:50.766650+0000","last_active":"2024-09-18T01:09:51.771446+0000","last_peered":"2024-09-18T01:09:51.771446+0000","last_clean":"2024-09-18T01:09:51.771446+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:09:51.771446+0000","last_undegraded":"2024-09-18T01:09:51.771446+0000","last_fullsized":"2024-09-18T01:09:51.771446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:55.743492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011506859999999999,"stat_sum":{"num_bytes":33871211,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4325,"num_read_kb":33197,"num_write":587,"num_write_kb":30063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1b","version":"158'1015","reported_seq":4877,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.956112+0000","last_change":"2024-09-18T01:09:50.885765+0000","last_active":"2024-09-18T01:09:51.956112+0000","last_peered":"2024-09-18T01:09:51.956112+0000","last_clean":"2024-09-18T01:09:51.956112+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:09:51.956112+0000","last_undegraded":"2024-09-18T01:09:51.956112+0000","last_fullsized":"2024-09-18T01:09:51.956112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:36.644518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078618199999999996,"stat_sum":{"num_bytes":31447143,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":7765,"num_read_kb":35162,"num_write":1514,"num_write_kb":28284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1a","version":"158'309","reported_seq":3644,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.708071+0000","last_change":"2024-09-18T01:09:50.762270+0000","last_active":"2024-09-18T01:09:55.708071+0000","last_peered":"2024-09-18T01:09:55.708071+0000","last_clean":"2024-09-18T01:09:55.708071+0000","last_became_active":"2024-09-18T01:09:23.609713+0000","last_became_peered":"2024-09-18T01:09:23.609713+0000","last_unstale":"2024-09-18T01:09:55.708071+0000","last_undegraded":"2024-09-18T01:09:55.708071+0000","last_fullsized":"2024-09-18T01:09:55.708071+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:45.902767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043256000000000002,"stat_sum":{"num_bytes":24195605,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3276,"num_read_kb":31059,"num_write":377,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":8984452,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.19","version":"158'287","reported_seq":5259,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.931838+0000","last_change":"2024-09-18T01:09:50.763562+0000","last_active":"2024-09-18T01:09:51.931838+0000","last_peered":"2024-09-18T01:09:51.931838+0000","last_clean":"2024-09-18T01:09:51.931838+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:09:51.931838+0000","last_undegraded":"2024-09-18T01:09:51.931838+0000","last_fullsized":"2024-09-18T01:09:51.931838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:22.893160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093155899999999999,"stat_sum":{"num_bytes":48842248,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4522,"num_read_kb":29589,"num_write":345,"num_write_kb":19103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.18","version":"158'359","reported_seq":10512,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564225+0000","last_change":"2024-09-18T01:09:50.768230+0000","last_active":"2024-09-18T01:09:54.564225+0000","last_peered":"2024-09-18T01:09:54.564225+0000","last_clean":"2024-09-18T01:09:54.564225+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:09:54.564225+0000","last_undegraded":"2024-09-18T01:09:54.564225+0000","last_fullsized":"2024-09-18T01:09:54.564225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:31.139215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049491499999999996,"stat_sum":{"num_bytes":53611503,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":10400,"num_read_kb":51652,"num_write":424,"num_write_kb":21999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.17","version":"158'617","reported_seq":4294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.905767+0000","last_change":"2024-09-18T01:09:50.886470+0000","last_active":"2024-09-18T01:09:51.905767+0000","last_peered":"2024-09-18T01:09:51.905767+0000","last_clean":"2024-09-18T01:09:51.905767+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:09:51.905767+0000","last_undegraded":"2024-09-18T01:09:51.905767+0000","last_fullsized":"2024-09-18T01:09:51.905767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:10.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571099999999999,"stat_sum":{"num_bytes":41105907,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4757,"num_read_kb":33007,"num_write":933,"num_write_kb":24143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.16","version":"158'512","reported_seq":2276,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.001615+0000","last_change":"2024-09-18T01:09:50.765641+0000","last_active":"2024-09-18T01:09:52.001615+0000","last_peered":"2024-09-18T01:09:52.001615+0000","last_clean":"2024-09-18T01:09:52.001615+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:09:52.001615+0000","last_undegraded":"2024-09-18T01:09:52.001615+0000","last_fullsized":"2024-09-18T01:09:52.001615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:10:27.915016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035700899999999998,"stat_sum":{"num_bytes":30635183,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2723,"num_read_kb":18698,"num_write":733,"num_write_kb":16761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.15","version":"158'645","reported_seq":7547,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.910825+0000","last_change":"2024-09-18T01:09:50.772001+0000","last_active":"2024-09-18T01:09:51.910825+0000","last_peered":"2024-09-18T01:09:51.910825+0000","last_clean":"2024-09-18T01:09:51.910825+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:09:51.910825+0000","last_undegraded":"2024-09-18T01:09:51.910825+0000","last_fullsized":"2024-09-18T01:09:51.910825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:31:07.949382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011871169999999999,"stat_sum":{"num_bytes":44942198,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8379,"num_read_kb":54393,"num_write":988,"num_write_kb":26785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.14","version":"158'541","reported_seq":4167,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.748721+0000","last_change":"2024-09-18T01:09:50.762330+0000","last_active":"2024-09-18T01:09:55.748721+0000","last_peered":"2024-09-18T01:09:55.748721+0000","last_clean":"2024-09-18T01:09:55.748721+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:09:55.748721+0000","last_undegraded":"2024-09-18T01:09:55.748721+0000","last_fullsized":"2024-09-18T01:09:55.748721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:12.182602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322336,"stat_sum":{"num_bytes":46085607,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4768,"num_read_kb":17696,"num_write":770,"num_write_kb":12186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.777222+0000","last_change":"2024-09-18T01:09:34.569456+0000","last_active":"2024-09-18T01:09:51.777222+0000","last_peered":"2024-09-18T01:09:51.777222+0000","last_clean":"2024-09-18T01:09:51.777222+0000","last_became_active":"2024-09-18T01:09:34.569116+0000","last_became_peered":"2024-09-18T01:09:34.569116+0000","last_unstale":"2024-09-18T01:09:51.777222+0000","last_undegraded":"2024-09-18T01:09:51.777222+0000","last_fullsized":"2024-09-18T01:09:51.777222+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:41.051534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"157'355","reported_seq":2809,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.775652+0000","last_change":"2024-09-18T01:09:50.886535+0000","last_active":"2024-09-18T01:09:51.775652+0000","last_peered":"2024-09-18T01:09:51.775652+0000","last_clean":"2024-09-18T01:09:51.775652+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:09:51.775652+0000","last_undegraded":"2024-09-18T01:09:51.775652+0000","last_fullsized":"2024-09-18T01:09:51.775652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:23.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297692,"stat_sum":{"num_bytes":13862072,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2988,"num_read_kb":20407,"num_write":496,"num_write_kb":17231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771692+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:09:51.771692+0000","last_peered":"2024-09-18T01:09:51.771692+0000","last_clean":"2024-09-18T01:09:51.771692+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:09:51.771692+0000","last_undegraded":"2024-09-18T01:09:51.771692+0000","last_fullsized":"2024-09-18T01:09:51.771692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"158'326","reported_seq":5810,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564071+0000","last_change":"2024-09-18T01:09:50.768784+0000","last_active":"2024-09-18T01:09:54.564071+0000","last_peered":"2024-09-18T01:09:54.564071+0000","last_clean":"2024-09-18T01:09:54.564071+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:09:54.564071+0000","last_undegraded":"2024-09-18T01:09:54.564071+0000","last_fullsized":"2024-09-18T01:09:54.564071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:59.005705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000803847,"stat_sum":{"num_bytes":31502903,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5563,"num_read_kb":27792,"num_write":498,"num_write_kb":21589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.773382+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:09:51.773382+0000","last_peered":"2024-09-18T01:09:51.773382+0000","last_clean":"2024-09-18T01:09:51.773382+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:09:51.773382+0000","last_undegraded":"2024-09-18T01:09:51.773382+0000","last_fullsized":"2024-09-18T01:09:51.773382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"157'275","reported_seq":3739,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771532+0000","last_change":"2024-09-18T01:09:50.763104+0000","last_active":"2024-09-18T01:09:51.771532+0000","last_peered":"2024-09-18T01:09:51.771532+0000","last_clean":"2024-09-18T01:09:51.771532+0000","last_became_active":"2024-09-18T01:09:37.673856+0000","last_became_peered":"2024-09-18T01:09:37.673856+0000","last_unstale":"2024-09-18T01:09:51.771532+0000","last_undegraded":"2024-09-18T01:09:51.771532+0000","last_fullsized":"2024-09-18T01:09:51.771532+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:21.099546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075023699999999995,"stat_sum":{"num_bytes":39804026,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3003,"num_read_kb":37047,"num_write":313,"num_write_kb":20895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":20418725,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771598+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:09:51.771598+0000","last_peered":"2024-09-18T01:09:51.771598+0000","last_clean":"2024-09-18T01:09:51.771598+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:09:51.771598+0000","last_undegraded":"2024-09-18T01:09:51.771598+0000","last_fullsized":"2024-09-18T01:09:51.771598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'736","reported_seq":10147,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.017873+0000","last_change":"2024-09-18T01:09:50.766516+0000","last_active":"2024-09-18T01:09:52.017873+0000","last_peered":"2024-09-18T01:09:52.017873+0000","last_clean":"2024-09-18T01:09:52.017873+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:09:52.017873+0000","last_undegraded":"2024-09-18T01:09:52.017873+0000","last_fullsized":"2024-09-18T01:09:52.017873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:28:34.737733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011794660000000001,"stat_sum":{"num_bytes":56967039,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":11765,"num_read_kb":55605,"num_write":1196,"num_write_kb":33369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.3","version":"158'355","reported_seq":5388,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.750204+0000","last_change":"2024-09-18T01:09:50.762524+0000","last_active":"2024-09-18T01:09:55.750204+0000","last_peered":"2024-09-18T01:09:55.750204+0000","last_clean":"2024-09-18T01:09:55.750204+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:09:55.750204+0000","last_undegraded":"2024-09-18T01:09:55.750204+0000","last_fullsized":"2024-09-18T01:09:55.750204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:30.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041215499999999999,"stat_sum":{"num_bytes":32040585,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5289,"num_read_kb":35220,"num_write":489,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":295,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771365+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:09:51.771365+0000","last_peered":"2024-09-18T01:09:51.771365+0000","last_clean":"2024-09-18T01:09:51.771365+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:09:51.771365+0000","last_undegraded":"2024-09-18T01:09:51.771365+0000","last_fullsized":"2024-09-18T01:09:51.771365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"158'419","reported_seq":9323,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.983549+0000","last_change":"2024-09-18T01:09:50.763220+0000","last_active":"2024-09-18T01:09:51.983549+0000","last_peered":"2024-09-18T01:09:51.983549+0000","last_clean":"2024-09-18T01:09:51.983549+0000","last_became_active":"2024-09-18T01:09:37.673616+0000","last_became_peered":"2024-09-18T01:09:37.673616+0000","last_unstale":"2024-09-18T01:09:51.983549+0000","last_undegraded":"2024-09-18T01:09:51.983549+0000","last_fullsized":"2024-09-18T01:09:51.983549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:50:16.043773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087490400000000002,"stat_sum":{"num_bytes":50023717,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9166,"num_read_kb":49966,"num_write":638,"num_write_kb":23800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":24181969,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771529+0000","last_change":"2024-09-18T01:09:34.566295+0000","last_active":"2024-09-18T01:09:51.771529+0000","last_peered":"2024-09-18T01:09:51.771529+0000","last_clean":"2024-09-18T01:09:51.771529+0000","last_became_active":"2024-09-18T01:09:34.565971+0000","last_became_peered":"2024-09-18T01:09:34.565971+0000","last_unstale":"2024-09-18T01:09:51.771529+0000","last_undegraded":"2024-09-18T01:09:51.771529+0000","last_fullsized":"2024-09-18T01:09:51.771529+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:05:01.087430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"158'414","reported_seq":11223,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.870017+0000","last_change":"2024-09-18T01:09:50.885725+0000","last_active":"2024-09-18T01:09:51.870017+0000","last_peered":"2024-09-18T01:09:51.870017+0000","last_clean":"2024-09-18T01:09:51.870017+0000","last_became_active":"2024-09-18T01:09:30.563706+0000","last_became_peered":"2024-09-18T01:09:30.563706+0000","last_unstale":"2024-09-18T01:09:51.870017+0000","last_undegraded":"2024-09-18T01:09:51.870017+0000","last_fullsized":"2024-09-18T01:09:51.870017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:11.986714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084249800000000001,"stat_sum":{"num_bytes":87706077,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":10736,"num_read_kb":34751,"num_write":493,"num_write_kb":31262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":6374523,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":293,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.772998+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:09:51.772998+0000","last_peered":"2024-09-18T01:09:51.772998+0000","last_clean":"2024-09-18T01:09:51.772998+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:09:51.772998+0000","last_undegraded":"2024-09-18T01:09:51.772998+0000","last_fullsized":"2024-09-18T01:09:51.772998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"158'319","reported_seq":4535,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.563759+0000","last_change":"2024-09-18T01:09:50.768454+0000","last_active":"2024-09-18T01:09:54.563759+0000","last_peered":"2024-09-18T01:09:54.563759+0000","last_clean":"2024-09-18T01:09:54.563759+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:09:54.563759+0000","last_undegraded":"2024-09-18T01:09:54.563759+0000","last_fullsized":"2024-09-18T01:09:54.563759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:07.796005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063912099999999998,"stat_sum":{"num_bytes":36692602,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4036,"num_read_kb":21239,"num_write":357,"num_write_kb":27502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":312,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771183+0000","last_change":"2024-09-18T01:09:34.576994+0000","last_active":"2024-09-18T01:09:51.771183+0000","last_peered":"2024-09-18T01:09:51.771183+0000","last_clean":"2024-09-18T01:09:51.771183+0000","last_became_active":"2024-09-18T01:09:34.571491+0000","last_became_peered":"2024-09-18T01:09:34.571491+0000","last_unstale":"2024-09-18T01:09:51.771183+0000","last_undegraded":"2024-09-18T01:09:51.771183+0000","last_fullsized":"2024-09-18T01:09:51.771183+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:15:32.312637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":445,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.773019+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:09:51.773019+0000","last_peered":"2024-09-18T01:09:51.773019+0000","last_clean":"2024-09-18T01:09:51.773019+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:09:51.773019+0000","last_undegraded":"2024-09-18T01:09:51.773019+0000","last_fullsized":"2024-09-18T01:09:51.773019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"158'641","reported_seq":2519,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.563602+0000","last_change":"2024-09-18T01:09:50.768236+0000","last_active":"2024-09-18T01:09:54.563602+0000","last_peered":"2024-09-18T01:09:54.563602+0000","last_clean":"2024-09-18T01:09:54.563602+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:09:54.563602+0000","last_undegraded":"2024-09-18T01:09:54.563602+0000","last_fullsized":"2024-09-18T01:09:54.563602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:52.986144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055222699999999997,"stat_sum":{"num_bytes":19944719,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3370,"num_read_kb":16410,"num_write":1097,"num_write_kb":13400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.9","version":"158'443","reported_seq":9969,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.931860+0000","last_change":"2024-09-18T01:09:50.762954+0000","last_active":"2024-09-18T01:09:51.931860+0000","last_peered":"2024-09-18T01:09:51.931860+0000","last_clean":"2024-09-18T01:09:51.931860+0000","last_became_active":"2024-09-18T01:09:37.678565+0000","last_became_peered":"2024-09-18T01:09:37.678565+0000","last_unstale":"2024-09-18T01:09:51.931860+0000","last_undegraded":"2024-09-18T01:09:51.931860+0000","last_fullsized":"2024-09-18T01:09:51.931860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:31.392440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000429522,"stat_sum":{"num_bytes":44675140,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":9758,"num_read_kb":39272,"num_write":622,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":25699382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.a","version":"158'342","reported_seq":9070,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.829364+0000","last_change":"2024-09-18T01:09:50.904453+0000","last_active":"2024-09-18T01:09:51.829364+0000","last_peered":"2024-09-18T01:09:51.829364+0000","last_clean":"2024-09-18T01:09:51.829364+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:09:51.829364+0000","last_undegraded":"2024-09-18T01:09:51.829364+0000","last_fullsized":"2024-09-18T01:09:51.829364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:21.148982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031910499999999997,"stat_sum":{"num_bytes":63874543,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8936,"num_read_kb":39440,"num_write":447,"num_write_kb":18194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.b","version":"158'481","reported_seq":3239,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564059+0000","last_change":"2024-09-18T01:09:50.768527+0000","last_active":"2024-09-18T01:09:54.564059+0000","last_peered":"2024-09-18T01:09:54.564059+0000","last_clean":"2024-09-18T01:09:54.564059+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:09:54.564059+0000","last_undegraded":"2024-09-18T01:09:54.564059+0000","last_fullsized":"2024-09-18T01:09:54.564059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:37.371316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065827400000000003,"stat_sum":{"num_bytes":20718782,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3643,"num_read_kb":25372,"num_write":669,"num_write_kb":19687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.c","version":"158'703","reported_seq":3324,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.987167+0000","last_change":"2024-09-18T01:09:50.765779+0000","last_active":"2024-09-18T01:09:51.987167+0000","last_peered":"2024-09-18T01:09:51.987167+0000","last_clean":"2024-09-18T01:09:51.987167+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:09:51.987167+0000","last_undegraded":"2024-09-18T01:09:51.987167+0000","last_fullsized":"2024-09-18T01:09:51.987167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:11.186519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049649099999999995,"stat_sum":{"num_bytes":19404233,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4397,"num_read_kb":25411,"num_write":1190,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.d","version":"158'384","reported_seq":12050,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.908209+0000","last_change":"2024-09-18T01:09:50.771262+0000","last_active":"2024-09-18T01:09:51.908209+0000","last_peered":"2024-09-18T01:09:51.908209+0000","last_clean":"2024-09-18T01:09:51.908209+0000","last_became_active":"2024-09-18T01:09:31.562903+0000","last_became_peered":"2024-09-18T01:09:31.562903+0000","last_unstale":"2024-09-18T01:09:51.908209+0000","last_undegraded":"2024-09-18T01:09:51.908209+0000","last_fullsized":"2024-09-18T01:09:51.908209+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:08.770350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056855600000000001,"stat_sum":{"num_bytes":59188400,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":12017,"num_read_kb":44929,"num_write":533,"num_write_kb":21530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":10117427,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.e","version":"158'587","reported_seq":5332,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.874489+0000","last_change":"2024-09-18T01:09:50.771305+0000","last_active":"2024-09-18T01:09:51.874489+0000","last_peered":"2024-09-18T01:09:51.874489+0000","last_clean":"2024-09-18T01:09:51.874489+0000","last_became_active":"2024-09-18T01:09:30.560764+0000","last_became_peered":"2024-09-18T01:09:30.560764+0000","last_unstale":"2024-09-18T01:09:51.874489+0000","last_undegraded":"2024-09-18T01:09:51.874489+0000","last_fullsized":"2024-09-18T01:09:51.874489+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.589861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054686999999999997,"stat_sum":{"num_bytes":40399376,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":5612,"num_read_kb":49951,"num_write":881,"num_write_kb":34623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":14011747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.f","version":"158'285","reported_seq":2764,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.922234+0000","last_change":"2024-09-18T01:09:50.771958+0000","last_active":"2024-09-18T01:09:51.922234+0000","last_peered":"2024-09-18T01:09:51.922234+0000","last_clean":"2024-09-18T01:09:51.922234+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:09:51.922234+0000","last_undegraded":"2024-09-18T01:09:51.922234+0000","last_fullsized":"2024-09-18T01:09:51.922234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:46.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038972,"stat_sum":{"num_bytes":45832042,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2291,"num_read_kb":23372,"num_write":409,"num_write_kb":14359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.10","version":"158'405","reported_seq":1708,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.001588+0000","last_change":"2024-09-18T01:09:50.904357+0000","last_active":"2024-09-18T01:09:52.001588+0000","last_peered":"2024-09-18T01:09:52.001588+0000","last_clean":"2024-09-18T01:09:52.001588+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:09:52.001588+0000","last_undegraded":"2024-09-18T01:09:52.001588+0000","last_fullsized":"2024-09-18T01:09:52.001588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:23.345119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000293763,"stat_sum":{"num_bytes":16477021,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1696,"num_read_kb":18019,"num_write":525,"num_write_kb":18639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.11","version":"158'290","reported_seq":2135,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.905512+0000","last_change":"2024-09-18T01:09:50.771039+0000","last_active":"2024-09-18T01:09:51.905512+0000","last_peered":"2024-09-18T01:09:51.905512+0000","last_clean":"2024-09-18T01:09:51.905512+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:09:51.905512+0000","last_undegraded":"2024-09-18T01:09:51.905512+0000","last_fullsized":"2024-09-18T01:09:51.905512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:30.780311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036113300000000003,"stat_sum":{"num_bytes":56711235,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1516,"num_read_kb":14090,"num_write":433,"num_write_kb":16302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.12","version":"158'1172","reported_seq":3805,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.010785+0000","last_change":"2024-09-18T01:09:50.886106+0000","last_active":"2024-09-18T01:09:52.010785+0000","last_peered":"2024-09-18T01:09:52.010785+0000","last_clean":"2024-09-18T01:09:52.010785+0000","last_became_active":"2024-09-18T01:09:31.564934+0000","last_became_peered":"2024-09-18T01:09:31.564934+0000","last_unstale":"2024-09-18T01:09:52.010785+0000","last_undegraded":"2024-09-18T01:09:52.010785+0000","last_fullsized":"2024-09-18T01:09:52.010785+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1172,"log_dups_size":0,"ondisk_log_size":1172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:09.706896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001013241,"stat_sum":{"num_bytes":20377752,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6552,"num_read_kb":24571,"num_write":2067,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2847823,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.13","version":"158'599","reported_seq":4081,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.922495+0000","last_change":"2024-09-18T01:09:50.771003+0000","last_active":"2024-09-18T01:09:51.922495+0000","last_peered":"2024-09-18T01:09:51.922495+0000","last_clean":"2024-09-18T01:09:51.922495+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:09:51.922495+0000","last_undegraded":"2024-09-18T01:09:51.922495+0000","last_fullsized":"2024-09-18T01:09:51.922495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:01.462015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041153899999999999,"stat_sum":{"num_bytes":31255668,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4972,"num_read_kb":15595,"num_write":909,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1237477908,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":175552,"num_read_kb":1011276,"num_write":22483,"num_write_kb":703164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":764,"num_bytes_recovered":112636048,"num_keys_recovered":99,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647081984,"data_stored":643116919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15352,"ondisk_log_size":15352,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163412,"kb_used_data":91972,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94208428,"statfs":{"total":96636764160,"available":96469430272,"internally_reserved":0,"allocated":94179328,"data_stored":92626986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87140,"kb_used_data":51636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94284700,"statfs":{"total":96636764160,"available":96547532800,"internally_reserved":0,"allocated":52875264,"data_stored":51623136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136040,"kb_used_data":90968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94235800,"statfs":{"total":96636764160,"available":96497459200,"internally_reserved":0,"allocated":93151232,"data_stored":91688181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74668,"kb_used_data":38328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94297172,"statfs":{"total":96636764160,"available":96560304128,"internally_reserved":0,"allocated":39247872,"data_stored":38029596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104864,"kb_used_data":64052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94266976,"statfs":{"total":96636764160,"available":96529383424,"internally_reserved":0,"allocated":65589248,"data_stored":64187987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171784,"kb_used_data":98668,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94200056,"statfs":{"total":96636764160,"available":96460857344,"internally_reserved":0,"allocated":101036032,"data_stored":99446532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147572,"kb_used_data":94176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94224268,"statfs":{"total":96636764160,"available":96485650432,"internally_reserved":0,"allocated":96436224,"data_stored":94869650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167888,"kb_used_data":117072,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94203952,"statfs":{"total":96636764160,"available":96464846848,"internally_reserved":0,"allocated":119881728,"data_stored":118096548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98549760,"data_stored":97941563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":63692800,"data_stored":63273367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37388288,"data_stored":37143527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":117964800,"data_stored":117169404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":95555584,"data_stored":94984945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":91254784,"data_stored":90773561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50982912,"data_stored":50708535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":91693056,"data_stored":91122017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:09:57.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:09:57.973 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:57.971+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:57.979 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:57.979 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:57.979 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:57.979 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:57.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:09:58.036 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:58.035+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:58.092 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:58.092 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:58.101 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:09:58.137 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:58.135+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:58.238 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:58.236+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:8397 3.5 3:a4afe4f7:::rbd_data.11762d3aac1.0000000000000005:head [write 0~3145728 [fadvise_sequential+fadvise_nocache],zero 3145728~524288,zero 4194304~114688,write 4308992~16384 [fadvise_sequential+fadvise_nocache],zero 4325376~114688,write 4440064~16384 [fadvise_sequential+fadvise_nocache],zero 4456448~114688,write 4571136~16384 [fadvise_sequential+fadvise_nocache],zero 4587520~114688,write 4702208~819200 [fadvise_sequential+fadvise_nocache],zero 5521408~49152,write 5570560~81920 [fadvise_sequential+fadvise_nocache],zero 5652480~49152,write 5701632~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 137.634790651, 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041498544999999998, 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 8397 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608288+0000", 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.608290+0000", 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617650+0000", 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0093602660000000008 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617666+0000", 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5815999999999999e-05 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617674+0000", 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4489999999999999e-06 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617736+0000", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1657999999999998e-05 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.617853+0000", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117454 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.618264+0000", 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041045699999999998 2024-09-18T01:09:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.625497+0000", 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072334160000000003 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649657+0000", 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024159711 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649727+0000", 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0457999999999995e-05 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.649787+0000", 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9552999999999997e-05 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:9170 3.1d 3:bb9636c1:::rbd_data.11762d3aac1.000000000000000c:head [write 0~478425 [fadvise_sequential+fadvise_nocache],zero 478425~45863,write 524288~524288 [fadvise_sequential+fadvise_nocache],zero 1048576~524288,write 1572864~115227 [fadvise_sequential+fadvise_nocache],zero 1688091~409061,write 2097152~1966080 [fadvise_sequential+fadvise_nocache],zero 4063232~114688,write 4177920~1064960 [fadvise_sequential+fadvise_nocache],zero 5767168~65536,write 5832704~32768 [fadvise_sequential+fadvise_nocache],zero 5865472~98304,write 5963776~32768 [fadvise_sequential+fadvise_nocache],zero 5996544~98304,write 6094848~32768 [fadvise_sequential+fadvise_nocache],zero 6127616~98304,write 6225920~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 137.58534475900001, 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034816001999999999, 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 9170 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657734+0000", 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.657735+0000", 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663645+0000", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059100380000000003 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663663+0000", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7878000000000001e-05 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663671+0000", 2024-09-18T01:09:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8699999999999992e-06 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663727+0000", 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5625999999999999e-05 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.663844+0000", 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000117322 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.664068+0000", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022389600000000001 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.682284+0000", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018216111 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692451+0000", 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010167229 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692504+0000", 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3247e-05 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:40.692550+0000", 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5822000000000003e-05 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:121600 3.5 3:a5560957:::rbd_header.1176e38d32d8:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.672933108999999, 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034695009999999998, 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 121600 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570146+0000", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570147+0000", 2024-09-18T01:09:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.85e-07 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570151+0000", 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570162+0000", 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0317e-05 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570167+0000", 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570207+0000", 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0543999999999997e-05 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570262+0000", 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4539000000000001e-05 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570403+0000", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140846 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.570909+0000", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050672900000000003 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604784+0000", 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033874597999999999 2024-09-18T01:09:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604816+0000", 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1668000000000002e-05 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:24.604841+0000", 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999999e-05 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186272 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758326+0000", 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4847533720000001, 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031399528000000003, 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186272 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758326+0000", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758326+0000", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758326+0000", 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2500000000000001e-07 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758336+0000", 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0409e-05 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758345+0000", 2024-09-18T01:09:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1969999999999996e-06 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758351+0000", 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.789422+0000", 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031070199999999999 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.789515+0000", 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3282999999999994e-05 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.789725+0000", 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021030300000000001 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186405 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 33792~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758394+0000", 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484684662, 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031658835000000003, 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186405 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758394+0000", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758394+0000", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758395+0000", 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9000000000000002e-07 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758398+0000", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9100000000000001e-06 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758405+0000", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7100000000000007e-06 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758411+0000", 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3820000000000003e-06 2024-09-18T01:09:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.789780+0000", 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031368788000000002 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.789835+0000", 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5352999999999997e-05 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790053+0000", 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000218302 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186406 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 72704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758417+0000", 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4846616100000001, 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031969170999999998, 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186406 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758417+0000", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758417+0000", 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758418+0000", 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1699999999999999e-07 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758420+0000", 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.305e-06 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758426+0000", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8050000000000003e-06 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758430+0000", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5820000000000003e-06 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790102+0000", 2024-09-18T01:09:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031671066999999997 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790185+0000", 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3146999999999995e-05 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790387+0000", 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000201848 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186407 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 78848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758437+0000", 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4846416339999999, 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032293059999999998, 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186407 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758437+0000", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758437+0000", 2024-09-18T01:09:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758438+0000", 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758440+0000", 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1220000000000002e-06 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758446+0000", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758451+0000", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-06 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790422+0000", 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031971152000000003 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790504+0000", 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1780000000000006e-05 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790730+0000", 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022682400000000001 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186408 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 84992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758455+0000", 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484623523, 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03258602, 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186408 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758455+0000", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758455+0000", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758456+0000", 2024-09-18T01:09:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758458+0000", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7489999999999999e-06 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758463+0000", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6969999999999998e-06 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758467+0000", 2024-09-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9609999999999999e-06 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790763+0000", 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032296104999999999 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.790841+0000", 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7260000000000002e-05 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791041+0000", 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200824 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186409 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 91136~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758473+0000", 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484605983, 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032938778000000002, 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186409 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758473+0000", 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758473+0000", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758473+0000", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758475+0000", 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0540000000000002e-06 2024-09-18T01:09:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758481+0000", 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.755e-06 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758486+0000", 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9289999999999997e-06 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791090+0000", 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032603655000000002 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791168+0000", 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8477000000000001e-05 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791412+0000", 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024349699999999999 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186410 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 130048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758491+0000", 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484587613, 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033240308000000003, 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186410 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758491+0000", 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758491+0000", 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758492+0000", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758493+0000", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8059999999999999e-06 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758509+0000", 2024-09-18T01:09:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6019999999999999e-05 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758514+0000", 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2719999999999999e-06 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791460+0000", 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032946527000000003 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791540+0000", 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9678000000000005e-05 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791732+0000", 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019172300000000001 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186411 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 136192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758519+0000", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484560409, 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034533833999999999, 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186411 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758519+0000", 2024-09-18T01:09:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758519+0000", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758519+0000", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4299999999999999e-07 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758521+0000", 2024-09-18T01:09:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.823e-06 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758525+0000", 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9389999999999999e-06 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758527+0000", 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.751e-06 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791783+0000", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.033255436999999999 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.791876+0000", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3308999999999997e-05 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793052+0000", 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001176332 2024-09-18T01:09:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186412 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 142336~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758530+0000", 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4845487399999999, 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034808197999999999, 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186412 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758530+0000", 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758530+0000", 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758530+0000", 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758532+0000", 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.229e-06 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758534+0000", 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7700000000000002e-06 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758537+0000", 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5859999999999999e-06 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793080+0000", 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034543068000000003 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793158+0000", 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8066999999999996e-05 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793338+0000", 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018025199999999999 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186413 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758540+0000", 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4845390700000001, 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035127991999999997, 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186413 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758540+0000", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758540+0000", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758540+0000", 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.05e-07 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758541+0000", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0899999999999999e-06 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758544+0000", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9289999999999998e-06 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758557+0000", 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3342e-05 2024-09-18T01:09:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793392+0000", 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034834885000000003 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793474+0000", 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2131000000000004e-05 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793668+0000", 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019341000000000001 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186414 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 187392~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758569+0000", 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4845099610000001, 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035463890999999997, 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186414 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758569+0000", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758569+0000", 2024-09-18T01:09:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758570+0000", 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758572+0000", 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4959999999999999e-06 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758578+0000", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3589999999999996e-06 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758582+0000", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7510000000000002e-06 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793699+0000", 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035117262000000003 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.793779+0000", 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9257000000000004e-05 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794033+0000", 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025428499999999998 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186415 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 193536~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758588+0000", 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484490756, 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035715202000000001, 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186415 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758588+0000", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758588+0000", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758589+0000", 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758601+0000", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1993e-05 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758608+0000", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9399999999999996e-06 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758613+0000", 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7540000000000003e-06 2024-09-18T01:09:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794062+0000", 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035448690999999997 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794108+0000", 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6310000000000002e-05 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794303+0000", 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195052 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186416 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 199680~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758620+0000", 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484458694, 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036054834000000001, 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186416 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758620+0000", 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758620+0000", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758621+0000", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758622+0000", 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7069999999999999e-06 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758628+0000", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2240000000000003e-06 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758632+0000", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3259999999999997e-06 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794349+0000", 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035716852 2024-09-18T01:09:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794436+0000", 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7570000000000004e-05 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794675+0000", 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023884399999999999 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186417 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 205824~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758638+0000", 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4844413359999999, 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036383163000000003, 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186417 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758638+0000", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758638+0000", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758638+0000", 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:09:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758640+0000", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758647+0000", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2829999999999998e-06 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758652+0000", 2024-09-18T01:09:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8459999999999999e-06 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794709+0000", 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036057201999999997 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.794788+0000", 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9503000000000001e-05 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795021+0000", 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023248999999999999 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186418 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 244736~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758657+0000", 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4844221690000001, 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036678495999999998, 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186418 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758657+0000", 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758657+0000", 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758657+0000", 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6300000000000001e-07 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758659+0000", 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4419999999999999e-06 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758662+0000", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4410000000000002e-06 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758665+0000", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9859999999999999e-06 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795060+0000", 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036395087999999999 2024-09-18T01:09:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795139+0000", 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9402999999999999e-05 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795335+0000", 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195873 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186419 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 250880~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758670+0000", 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.484409114, 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036927968999999998, 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186419 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758670+0000", 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758670+0000", 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758670+0000", 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758672+0000", 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.818e-06 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758675+0000", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3900000000000002e-06 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758679+0000", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2289999999999999e-06 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795358+0000", 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036679718 2024-09-18T01:09:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795405+0000", 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6884e-05 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795598+0000", 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019258 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186420 3.16 3:686fb719:::rbd_data.11762ddd5718.000000000000001d:48 [read 257024~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.758685+0000", 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.4843936879999999, 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037239546999999998, 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186420 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758685+0000", 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758685+0000", 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758686+0000", 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758688+0000", 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0540000000000002e-06 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758694+0000", 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2480000000000001e-06 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.758699+0000", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0340000000000004e-06 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795629+0000", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036930165000000001 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795712+0000", 2024-09-18T01:09:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2782999999999997e-05 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.795925+0000", 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021289400000000001 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.343 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:58.345 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:09:58.346 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:09:58.346 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:09:58.347 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:58.343+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:58.348 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-18T01:09:57.571213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1238068295,"num_objects":1300,"num_object_clones":0,"num_object_copies":2600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1300,"num_whiteouts":0,"num_read":175669,"num_read_kb":1011377,"num_write":22644,"num_write_kb":705582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":766,"num_bytes_recovered":112636048,"num_keys_recovered":99,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15472,"ondisk_log_size":15472,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1053368,"kb_used_data":646872,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":753921352,"statfs":{"total":773094113280,"available":772015464448,"internally_reserved":0,"allocated":662396928,"data_stored":650568616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":410242608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2485753,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":266,"num_read_kb":4258,"num_write":195,"num_write_kb":4926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.782525"},"pg_stats":[{"pgid":"3.1f","version":"158'359","reported_seq":3763,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.987343+0000","last_change":"2024-09-18T01:09:50.886020+0000","last_active":"2024-09-18T01:09:51.987343+0000","last_peered":"2024-09-18T01:09:51.987343+0000","last_clean":"2024-09-18T01:09:51.987343+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:09:51.987343+0000","last_undegraded":"2024-09-18T01:09:51.987343+0000","last_fullsized":"2024-09-18T01:09:51.987343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:59:33.823603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010689580000000001,"stat_sum":{"num_bytes":41730540,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3495,"num_read_kb":30759,"num_write":398,"num_write_kb":27187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1e","version":"158'434","reported_seq":2130,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564078+0000","last_change":"2024-09-18T01:09:50.768682+0000","last_active":"2024-09-18T01:09:54.564078+0000","last_peered":"2024-09-18T01:09:54.564078+0000","last_clean":"2024-09-18T01:09:54.564078+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:09:54.564078+0000","last_undegraded":"2024-09-18T01:09:54.564078+0000","last_fullsized":"2024-09-18T01:09:54.564078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:50.620438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072675599999999995,"stat_sum":{"num_bytes":19075625,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2207,"num_read_kb":25460,"num_write":686,"num_write_kb":19903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1d","version":"157'324","reported_seq":2213,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771331+0000","last_change":"2024-09-18T01:09:50.765539+0000","last_active":"2024-09-18T01:09:51.771331+0000","last_peered":"2024-09-18T01:09:51.771331+0000","last_clean":"2024-09-18T01:09:51.771331+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:09:51.771331+0000","last_undegraded":"2024-09-18T01:09:51.771331+0000","last_fullsized":"2024-09-18T01:09:51.771331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:40.822510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031285699999999999,"stat_sum":{"num_bytes":34477206,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1629,"num_read_kb":32145,"num_write":466,"num_write_kb":36459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1c","version":"158'419","reported_seq":4301,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771446+0000","last_change":"2024-09-18T01:09:50.766650+0000","last_active":"2024-09-18T01:09:51.771446+0000","last_peered":"2024-09-18T01:09:51.771446+0000","last_clean":"2024-09-18T01:09:51.771446+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:09:51.771446+0000","last_undegraded":"2024-09-18T01:09:51.771446+0000","last_fullsized":"2024-09-18T01:09:51.771446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:55.743492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011506859999999999,"stat_sum":{"num_bytes":33871211,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4325,"num_read_kb":33197,"num_write":587,"num_write_kb":30063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1b","version":"158'1015","reported_seq":4877,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.956112+0000","last_change":"2024-09-18T01:09:50.885765+0000","last_active":"2024-09-18T01:09:51.956112+0000","last_peered":"2024-09-18T01:09:51.956112+0000","last_clean":"2024-09-18T01:09:51.956112+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:09:51.956112+0000","last_undegraded":"2024-09-18T01:09:51.956112+0000","last_fullsized":"2024-09-18T01:09:51.956112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:36.644518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078618199999999996,"stat_sum":{"num_bytes":31447143,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":7765,"num_read_kb":35162,"num_write":1514,"num_write_kb":28284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.1a","version":"158'309","reported_seq":3644,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.708071+0000","last_change":"2024-09-18T01:09:50.762270+0000","last_active":"2024-09-18T01:09:55.708071+0000","last_peered":"2024-09-18T01:09:55.708071+0000","last_clean":"2024-09-18T01:09:55.708071+0000","last_became_active":"2024-09-18T01:09:23.609713+0000","last_became_peered":"2024-09-18T01:09:23.609713+0000","last_unstale":"2024-09-18T01:09:55.708071+0000","last_undegraded":"2024-09-18T01:09:55.708071+0000","last_fullsized":"2024-09-18T01:09:55.708071+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:45.902767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043256000000000002,"stat_sum":{"num_bytes":24195605,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3276,"num_read_kb":31059,"num_write":377,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":8984452,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.19","version":"158'287","reported_seq":5259,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.931838+0000","last_change":"2024-09-18T01:09:50.763562+0000","last_active":"2024-09-18T01:09:51.931838+0000","last_peered":"2024-09-18T01:09:51.931838+0000","last_clean":"2024-09-18T01:09:51.931838+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:09:51.931838+0000","last_undegraded":"2024-09-18T01:09:51.931838+0000","last_fullsized":"2024-09-18T01:09:51.931838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:22.893160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093155899999999999,"stat_sum":{"num_bytes":48842248,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4522,"num_read_kb":29589,"num_write":345,"num_write_kb":19103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.18","version":"158'359","reported_seq":10512,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564225+0000","last_change":"2024-09-18T01:09:50.768230+0000","last_active":"2024-09-18T01:09:54.564225+0000","last_peered":"2024-09-18T01:09:54.564225+0000","last_clean":"2024-09-18T01:09:54.564225+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:09:54.564225+0000","last_undegraded":"2024-09-18T01:09:54.564225+0000","last_fullsized":"2024-09-18T01:09:54.564225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:31.139215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049491499999999996,"stat_sum":{"num_bytes":53611503,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":10400,"num_read_kb":51652,"num_write":424,"num_write_kb":21999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.17","version":"158'617","reported_seq":4294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.905767+0000","last_change":"2024-09-18T01:09:50.886470+0000","last_active":"2024-09-18T01:09:51.905767+0000","last_peered":"2024-09-18T01:09:51.905767+0000","last_clean":"2024-09-18T01:09:51.905767+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:09:51.905767+0000","last_undegraded":"2024-09-18T01:09:51.905767+0000","last_fullsized":"2024-09-18T01:09:51.905767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:10.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571099999999999,"stat_sum":{"num_bytes":41105907,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4757,"num_read_kb":33007,"num_write":933,"num_write_kb":24143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.16","version":"158'512","reported_seq":2276,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.001615+0000","last_change":"2024-09-18T01:09:50.765641+0000","last_active":"2024-09-18T01:09:52.001615+0000","last_peered":"2024-09-18T01:09:52.001615+0000","last_clean":"2024-09-18T01:09:52.001615+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:09:52.001615+0000","last_undegraded":"2024-09-18T01:09:52.001615+0000","last_fullsized":"2024-09-18T01:09:52.001615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:10:27.915016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035700899999999998,"stat_sum":{"num_bytes":30635183,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2723,"num_read_kb":18698,"num_write":733,"num_write_kb":16761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.15","version":"158'645","reported_seq":7547,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.910825+0000","last_change":"2024-09-18T01:09:50.772001+0000","last_active":"2024-09-18T01:09:51.910825+0000","last_peered":"2024-09-18T01:09:51.910825+0000","last_clean":"2024-09-18T01:09:51.910825+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:09:51.910825+0000","last_undegraded":"2024-09-18T01:09:51.910825+0000","last_fullsized":"2024-09-18T01:09:51.910825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:31:07.949382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011871169999999999,"stat_sum":{"num_bytes":44942198,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8379,"num_read_kb":54393,"num_write":988,"num_write_kb":26785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.14","version":"158'541","reported_seq":4167,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.748721+0000","last_change":"2024-09-18T01:09:50.762330+0000","last_active":"2024-09-18T01:09:55.748721+0000","last_peered":"2024-09-18T01:09:55.748721+0000","last_clean":"2024-09-18T01:09:55.748721+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:09:55.748721+0000","last_undegraded":"2024-09-18T01:09:55.748721+0000","last_fullsized":"2024-09-18T01:09:55.748721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:12.182602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322336,"stat_sum":{"num_bytes":46085607,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4768,"num_read_kb":17696,"num_write":770,"num_write_kb":12186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.777222+0000","last_change":"2024-09-18T01:09:34.569456+0000","last_active":"2024-09-18T01:09:51.777222+0000","last_peered":"2024-09-18T01:09:51.777222+0000","last_clean":"2024-09-18T01:09:51.777222+0000","last_became_active":"2024-09-18T01:09:34.569116+0000","last_became_peered":"2024-09-18T01:09:34.569116+0000","last_unstale":"2024-09-18T01:09:51.777222+0000","last_undegraded":"2024-09-18T01:09:51.777222+0000","last_fullsized":"2024-09-18T01:09:51.777222+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:41.051534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"157'355","reported_seq":2809,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.775652+0000","last_change":"2024-09-18T01:09:50.886535+0000","last_active":"2024-09-18T01:09:51.775652+0000","last_peered":"2024-09-18T01:09:51.775652+0000","last_clean":"2024-09-18T01:09:51.775652+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:09:51.775652+0000","last_undegraded":"2024-09-18T01:09:51.775652+0000","last_fullsized":"2024-09-18T01:09:51.775652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:23.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297692,"stat_sum":{"num_bytes":13862072,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2988,"num_read_kb":20407,"num_write":496,"num_write_kb":17231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771692+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:09:51.771692+0000","last_peered":"2024-09-18T01:09:51.771692+0000","last_clean":"2024-09-18T01:09:51.771692+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:09:51.771692+0000","last_undegraded":"2024-09-18T01:09:51.771692+0000","last_fullsized":"2024-09-18T01:09:51.771692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"158'326","reported_seq":5810,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564071+0000","last_change":"2024-09-18T01:09:50.768784+0000","last_active":"2024-09-18T01:09:54.564071+0000","last_peered":"2024-09-18T01:09:54.564071+0000","last_clean":"2024-09-18T01:09:54.564071+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:09:54.564071+0000","last_undegraded":"2024-09-18T01:09:54.564071+0000","last_fullsized":"2024-09-18T01:09:54.564071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:59.005705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000803847,"stat_sum":{"num_bytes":31502903,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5563,"num_read_kb":27792,"num_write":498,"num_write_kb":21589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.773382+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:09:51.773382+0000","last_peered":"2024-09-18T01:09:51.773382+0000","last_clean":"2024-09-18T01:09:51.773382+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:09:51.773382+0000","last_undegraded":"2024-09-18T01:09:51.773382+0000","last_fullsized":"2024-09-18T01:09:51.773382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"157'275","reported_seq":3739,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771532+0000","last_change":"2024-09-18T01:09:50.763104+0000","last_active":"2024-09-18T01:09:51.771532+0000","last_peered":"2024-09-18T01:09:51.771532+0000","last_clean":"2024-09-18T01:09:51.771532+0000","last_became_active":"2024-09-18T01:09:37.673856+0000","last_became_peered":"2024-09-18T01:09:37.673856+0000","last_unstale":"2024-09-18T01:09:51.771532+0000","last_undegraded":"2024-09-18T01:09:51.771532+0000","last_fullsized":"2024-09-18T01:09:51.771532+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:21.099546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075023699999999995,"stat_sum":{"num_bytes":39804026,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3003,"num_read_kb":37047,"num_write":313,"num_write_kb":20895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":20418725,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":294,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771598+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:09:51.771598+0000","last_peered":"2024-09-18T01:09:51.771598+0000","last_clean":"2024-09-18T01:09:51.771598+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:09:51.771598+0000","last_undegraded":"2024-09-18T01:09:51.771598+0000","last_fullsized":"2024-09-18T01:09:51.771598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'736","reported_seq":10147,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.017873+0000","last_change":"2024-09-18T01:09:50.766516+0000","last_active":"2024-09-18T01:09:52.017873+0000","last_peered":"2024-09-18T01:09:52.017873+0000","last_clean":"2024-09-18T01:09:52.017873+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:09:52.017873+0000","last_undegraded":"2024-09-18T01:09:52.017873+0000","last_fullsized":"2024-09-18T01:09:52.017873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:28:34.737733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011794660000000001,"stat_sum":{"num_bytes":56967039,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":11765,"num_read_kb":55605,"num_write":1196,"num_write_kb":33369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.3","version":"158'355","reported_seq":5388,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T01:09:55.750204+0000","last_change":"2024-09-18T01:09:50.762524+0000","last_active":"2024-09-18T01:09:55.750204+0000","last_peered":"2024-09-18T01:09:55.750204+0000","last_clean":"2024-09-18T01:09:55.750204+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:09:55.750204+0000","last_undegraded":"2024-09-18T01:09:55.750204+0000","last_fullsized":"2024-09-18T01:09:55.750204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:30.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041215499999999999,"stat_sum":{"num_bytes":32040585,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5289,"num_read_kb":35220,"num_write":489,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":295,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771365+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:09:51.771365+0000","last_peered":"2024-09-18T01:09:51.771365+0000","last_clean":"2024-09-18T01:09:51.771365+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:09:51.771365+0000","last_undegraded":"2024-09-18T01:09:51.771365+0000","last_fullsized":"2024-09-18T01:09:51.771365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"158'419","reported_seq":9323,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.983549+0000","last_change":"2024-09-18T01:09:50.763220+0000","last_active":"2024-09-18T01:09:51.983549+0000","last_peered":"2024-09-18T01:09:51.983549+0000","last_clean":"2024-09-18T01:09:51.983549+0000","last_became_active":"2024-09-18T01:09:37.673616+0000","last_became_peered":"2024-09-18T01:09:37.673616+0000","last_unstale":"2024-09-18T01:09:51.983549+0000","last_undegraded":"2024-09-18T01:09:51.983549+0000","last_fullsized":"2024-09-18T01:09:51.983549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:50:16.043773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087490400000000002,"stat_sum":{"num_bytes":50023717,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9166,"num_read_kb":49966,"num_write":638,"num_write_kb":23800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":24181969,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771529+0000","last_change":"2024-09-18T01:09:34.566295+0000","last_active":"2024-09-18T01:09:51.771529+0000","last_peered":"2024-09-18T01:09:51.771529+0000","last_clean":"2024-09-18T01:09:51.771529+0000","last_became_active":"2024-09-18T01:09:34.565971+0000","last_became_peered":"2024-09-18T01:09:34.565971+0000","last_unstale":"2024-09-18T01:09:51.771529+0000","last_undegraded":"2024-09-18T01:09:51.771529+0000","last_fullsized":"2024-09-18T01:09:51.771529+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:05:01.087430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"158'414","reported_seq":11223,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.870017+0000","last_change":"2024-09-18T01:09:50.885725+0000","last_active":"2024-09-18T01:09:51.870017+0000","last_peered":"2024-09-18T01:09:51.870017+0000","last_clean":"2024-09-18T01:09:51.870017+0000","last_became_active":"2024-09-18T01:09:30.563706+0000","last_became_peered":"2024-09-18T01:09:30.563706+0000","last_unstale":"2024-09-18T01:09:51.870017+0000","last_undegraded":"2024-09-18T01:09:51.870017+0000","last_fullsized":"2024-09-18T01:09:51.870017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:11.986714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084249800000000001,"stat_sum":{"num_bytes":87706077,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":10736,"num_read_kb":34751,"num_write":493,"num_write_kb":31262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":6374523,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":293,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.772998+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:09:51.772998+0000","last_peered":"2024-09-18T01:09:51.772998+0000","last_clean":"2024-09-18T01:09:51.772998+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:09:51.772998+0000","last_undegraded":"2024-09-18T01:09:51.772998+0000","last_fullsized":"2024-09-18T01:09:51.772998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"158'319","reported_seq":4535,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.563759+0000","last_change":"2024-09-18T01:09:50.768454+0000","last_active":"2024-09-18T01:09:54.563759+0000","last_peered":"2024-09-18T01:09:54.563759+0000","last_clean":"2024-09-18T01:09:54.563759+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:09:54.563759+0000","last_undegraded":"2024-09-18T01:09:54.563759+0000","last_fullsized":"2024-09-18T01:09:54.563759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:07.796005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063912099999999998,"stat_sum":{"num_bytes":36692602,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4036,"num_read_kb":21239,"num_write":357,"num_write_kb":27502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":312,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.771183+0000","last_change":"2024-09-18T01:09:34.576994+0000","last_active":"2024-09-18T01:09:51.771183+0000","last_peered":"2024-09-18T01:09:51.771183+0000","last_clean":"2024-09-18T01:09:51.771183+0000","last_became_active":"2024-09-18T01:09:34.571491+0000","last_became_peered":"2024-09-18T01:09:34.571491+0000","last_unstale":"2024-09-18T01:09:51.771183+0000","last_undegraded":"2024-09-18T01:09:51.771183+0000","last_fullsized":"2024-09-18T01:09:51.771183+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:15:32.312637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":445,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.773019+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:09:51.773019+0000","last_peered":"2024-09-18T01:09:51.773019+0000","last_clean":"2024-09-18T01:09:51.773019+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:09:51.773019+0000","last_undegraded":"2024-09-18T01:09:51.773019+0000","last_fullsized":"2024-09-18T01:09:51.773019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"158'641","reported_seq":2519,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.563602+0000","last_change":"2024-09-18T01:09:50.768236+0000","last_active":"2024-09-18T01:09:54.563602+0000","last_peered":"2024-09-18T01:09:54.563602+0000","last_clean":"2024-09-18T01:09:54.563602+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:09:54.563602+0000","last_undegraded":"2024-09-18T01:09:54.563602+0000","last_fullsized":"2024-09-18T01:09:54.563602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:52.986144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055222699999999997,"stat_sum":{"num_bytes":19944719,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3370,"num_read_kb":16410,"num_write":1097,"num_write_kb":13400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.9","version":"158'443","reported_seq":9969,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.931860+0000","last_change":"2024-09-18T01:09:50.762954+0000","last_active":"2024-09-18T01:09:51.931860+0000","last_peered":"2024-09-18T01:09:51.931860+0000","last_clean":"2024-09-18T01:09:51.931860+0000","last_became_active":"2024-09-18T01:09:37.678565+0000","last_became_peered":"2024-09-18T01:09:37.678565+0000","last_unstale":"2024-09-18T01:09:51.931860+0000","last_undegraded":"2024-09-18T01:09:51.931860+0000","last_fullsized":"2024-09-18T01:09:51.931860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:31.392440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000429522,"stat_sum":{"num_bytes":44675140,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":9758,"num_read_kb":39272,"num_write":622,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":25699382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.a","version":"158'342","reported_seq":9070,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.829364+0000","last_change":"2024-09-18T01:09:50.904453+0000","last_active":"2024-09-18T01:09:51.829364+0000","last_peered":"2024-09-18T01:09:51.829364+0000","last_clean":"2024-09-18T01:09:51.829364+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:09:51.829364+0000","last_undegraded":"2024-09-18T01:09:51.829364+0000","last_fullsized":"2024-09-18T01:09:51.829364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:21.148982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031910499999999997,"stat_sum":{"num_bytes":63874543,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8936,"num_read_kb":39440,"num_write":447,"num_write_kb":18194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.b","version":"158'481","reported_seq":3239,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T01:09:54.564059+0000","last_change":"2024-09-18T01:09:50.768527+0000","last_active":"2024-09-18T01:09:54.564059+0000","last_peered":"2024-09-18T01:09:54.564059+0000","last_clean":"2024-09-18T01:09:54.564059+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:09:54.564059+0000","last_undegraded":"2024-09-18T01:09:54.564059+0000","last_fullsized":"2024-09-18T01:09:54.564059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:37.371316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065827400000000003,"stat_sum":{"num_bytes":20718782,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3643,"num_read_kb":25372,"num_write":669,"num_write_kb":19687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.c","version":"158'703","reported_seq":3324,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.987167+0000","last_change":"2024-09-18T01:09:50.765779+0000","last_active":"2024-09-18T01:09:51.987167+0000","last_peered":"2024-09-18T01:09:51.987167+0000","last_clean":"2024-09-18T01:09:51.987167+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:09:51.987167+0000","last_undegraded":"2024-09-18T01:09:51.987167+0000","last_fullsized":"2024-09-18T01:09:51.987167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:11.186519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049649099999999995,"stat_sum":{"num_bytes":19404233,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4397,"num_read_kb":25411,"num_write":1190,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.d","version":"158'384","reported_seq":12050,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.908209+0000","last_change":"2024-09-18T01:09:50.771262+0000","last_active":"2024-09-18T01:09:51.908209+0000","last_peered":"2024-09-18T01:09:51.908209+0000","last_clean":"2024-09-18T01:09:51.908209+0000","last_became_active":"2024-09-18T01:09:31.562903+0000","last_became_peered":"2024-09-18T01:09:31.562903+0000","last_unstale":"2024-09-18T01:09:51.908209+0000","last_undegraded":"2024-09-18T01:09:51.908209+0000","last_fullsized":"2024-09-18T01:09:51.908209+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:08.770350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056855600000000001,"stat_sum":{"num_bytes":59188400,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":12017,"num_read_kb":44929,"num_write":533,"num_write_kb":21530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":10117427,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.e","version":"158'587","reported_seq":5332,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.874489+0000","last_change":"2024-09-18T01:09:50.771305+0000","last_active":"2024-09-18T01:09:51.874489+0000","last_peered":"2024-09-18T01:09:51.874489+0000","last_clean":"2024-09-18T01:09:51.874489+0000","last_became_active":"2024-09-18T01:09:30.560764+0000","last_became_peered":"2024-09-18T01:09:30.560764+0000","last_unstale":"2024-09-18T01:09:51.874489+0000","last_undegraded":"2024-09-18T01:09:51.874489+0000","last_fullsized":"2024-09-18T01:09:51.874489+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.589861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054686999999999997,"stat_sum":{"num_bytes":40399376,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":5612,"num_read_kb":49951,"num_write":881,"num_write_kb":34623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":14011747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.f","version":"158'285","reported_seq":2764,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.922234+0000","last_change":"2024-09-18T01:09:50.771958+0000","last_active":"2024-09-18T01:09:51.922234+0000","last_peered":"2024-09-18T01:09:51.922234+0000","last_clean":"2024-09-18T01:09:51.922234+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:09:51.922234+0000","last_undegraded":"2024-09-18T01:09:51.922234+0000","last_fullsized":"2024-09-18T01:09:51.922234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:46.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038972,"stat_sum":{"num_bytes":45832042,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2291,"num_read_kb":23372,"num_write":409,"num_write_kb":14359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.10","version":"158'405","reported_seq":1708,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.001588+0000","last_change":"2024-09-18T01:09:50.904357+0000","last_active":"2024-09-18T01:09:52.001588+0000","last_peered":"2024-09-18T01:09:52.001588+0000","last_clean":"2024-09-18T01:09:52.001588+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:09:52.001588+0000","last_undegraded":"2024-09-18T01:09:52.001588+0000","last_fullsized":"2024-09-18T01:09:52.001588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:23.345119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000293763,"stat_sum":{"num_bytes":16477021,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1696,"num_read_kb":18019,"num_write":525,"num_write_kb":18639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.11","version":"158'290","reported_seq":2135,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.905512+0000","last_change":"2024-09-18T01:09:50.771039+0000","last_active":"2024-09-18T01:09:51.905512+0000","last_peered":"2024-09-18T01:09:51.905512+0000","last_clean":"2024-09-18T01:09:51.905512+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:09:51.905512+0000","last_undegraded":"2024-09-18T01:09:51.905512+0000","last_fullsized":"2024-09-18T01:09:51.905512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:30.780311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036113300000000003,"stat_sum":{"num_bytes":56711235,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1516,"num_read_kb":14090,"num_write":433,"num_write_kb":16302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.12","version":"158'1172","reported_seq":3805,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:52.010785+0000","last_change":"2024-09-18T01:09:50.886106+0000","last_active":"2024-09-18T01:09:52.010785+0000","last_peered":"2024-09-18T01:09:52.010785+0000","last_clean":"2024-09-18T01:09:52.010785+0000","last_became_active":"2024-09-18T01:09:31.564934+0000","last_became_peered":"2024-09-18T01:09:31.564934+0000","last_unstale":"2024-09-18T01:09:52.010785+0000","last_undegraded":"2024-09-18T01:09:52.010785+0000","last_fullsized":"2024-09-18T01:09:52.010785+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1172,"log_dups_size":0,"ondisk_log_size":1172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:09.706896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001013241,"stat_sum":{"num_bytes":20377752,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6552,"num_read_kb":24571,"num_write":2067,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2847823,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]},{"pgid":"3.13","version":"158'599","reported_seq":4081,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-18T01:09:51.922495+0000","last_change":"2024-09-18T01:09:50.771003+0000","last_active":"2024-09-18T01:09:51.922495+0000","last_peered":"2024-09-18T01:09:51.922495+0000","last_clean":"2024-09-18T01:09:51.922495+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:09:51.922495+0000","last_undegraded":"2024-09-18T01:09:51.922495+0000","last_fullsized":"2024-09-18T01:09:51.922495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:01.462015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041153899999999999,"stat_sum":{"num_bytes":31255668,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4972,"num_read_kb":15595,"num_write":909,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"43","length":"1"},{"start":"45","length":"1"},{"start":"47","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1237477908,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":175552,"num_read_kb":1011276,"num_write":22483,"num_write_kb":703164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":764,"num_bytes_recovered":112636048,"num_keys_recovered":99,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647081984,"data_stored":643116919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15352,"ondisk_log_size":15352,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163412,"kb_used_data":91972,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94208428,"statfs":{"total":96636764160,"available":96469430272,"internally_reserved":0,"allocated":94179328,"data_stored":92626986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87140,"kb_used_data":51636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94284700,"statfs":{"total":96636764160,"available":96547532800,"internally_reserved":0,"allocated":52875264,"data_stored":51623136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136040,"kb_used_data":90968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94235800,"statfs":{"total":96636764160,"available":96497459200,"internally_reserved":0,"allocated":93151232,"data_stored":91688181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74668,"kb_used_data":38328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94297172,"statfs":{"total":96636764160,"available":96560304128,"internally_reserved":0,"allocated":39247872,"data_stored":38029596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104864,"kb_used_data":64052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94266976,"statfs":{"total":96636764160,"available":96529383424,"internally_reserved":0,"allocated":65589248,"data_stored":64187987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171784,"kb_used_data":98668,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94200056,"statfs":{"total":96636764160,"available":96460857344,"internally_reserved":0,"allocated":101036032,"data_stored":99446532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147572,"kb_used_data":94176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94224268,"statfs":{"total":96636764160,"available":96485650432,"internally_reserved":0,"allocated":96436224,"data_stored":94869650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167888,"kb_used_data":117072,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94203952,"statfs":{"total":96636764160,"available":96464846848,"internally_reserved":0,"allocated":119881728,"data_stored":118096548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98549760,"data_stored":97941563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":63692800,"data_stored":63273367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37388288,"data_stored":37143527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":117964800,"data_stored":117169404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":95555584,"data_stored":94984945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":91254784,"data_stored":90773561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50982912,"data_stored":50708535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":91693056,"data_stored":91122017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:09:58.350 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:09:58.350 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:09:58.350 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-18T01:09:58.350 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:09:58.350 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:09:58.351 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-18T01:09:58.446 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:58.445+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:58.491 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:58.492 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:58.492 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:09:58.492 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:09:58.492 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:09:58.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:09:58.584 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:58.583+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:58.648 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:58.647+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:58.694 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:09:58.694 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:09:58.694 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:58.706 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:09:58.748 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:58.747+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:58.758 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:09:58.849 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:09:58.847+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:58.909 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.335122317, 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:09:58.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:09:58.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:09:58.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.283131174000005, 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:09:58.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:09:58.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.283054378000003, 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:09:58.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.278549225000006, 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:09:58.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:09:58.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:09:58.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.278293429000001, 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.274268101000004, 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:09:58.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.274126951, 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:09:58.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.273894046999999, 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:09:58.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.951 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.271274985000005, 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:09:58.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.248935950999993, 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:09:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:09:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298420676999996, 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:09:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298398935999998, 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:09:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298378528000001, 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:09:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298358868999998, 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:09:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:09:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298325534, 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:09:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:09:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298291001999999, 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:09:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:09:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:09:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298246112000001, 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:09:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298219736999997, 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:09:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298200585999993, 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:09:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.298182018000006, 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:09:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:09:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:09:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:09:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:09:59.010 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:59.010 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:09:59.011 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:09:59.012 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:09:59.012 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:09:59.012 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:09:59.013 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:59.010+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:59.044 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:09:59.045 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:09:59.097 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:09:59.097 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:09:59.113 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:09:59.111+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:09:59.131 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:09:59.131 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:09:59.159 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:09:59.159 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:09:59.189 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:09:59.189 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:09:59.205 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:09:59.205 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:09:59.205 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:09:59.206 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:09:59.213 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:09:59.212+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:09:59.219 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:09:59.221 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:09:59.221 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:09:59.314 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:59.312+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:59.414 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:09:59.413+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:09:59.422 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:59.422 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:59.434 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:09:59.523 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:09:59.521+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:09:59.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:09:59.616 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:59.615+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:59.635 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:59.635 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:59.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:09:59.760 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:09:59.715+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:09:59.802 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:59.802 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:59.811 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:09:59.818 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:09:59.817+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:09:59.919 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:09:59.916+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:09:59.970 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:09:59.970 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:09:59.980 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:00.023 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:00.019+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:00.120 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:00.118+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:00.152 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.153 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:00.245 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:00.243+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:00.321 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.321 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.324 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:00.321+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:00.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:00.424 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:00.423+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:00.478 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.478 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.487 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:00.524 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:00.523+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:10:00.608 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.608 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:00.625 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:00.624+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:00.726 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:00.725+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:00.765 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.765 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:00.826 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:00.825+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:00.917 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:00.917 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:00.927 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:00.925+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:00.929 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:01.028 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:01.026+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:01.109 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:01.109 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:01.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:01.158 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:01.156+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:01.229 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:01.227+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:01.236 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:01.262 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:01.262 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:01.271 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:01.329 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:01.328+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:01.358 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:01.358 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:01.374 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:01.375 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:01.391 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:01.391 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:01.407 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:01.407 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:01.418 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:01.418 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:01.418 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:01.424 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:01.424 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:01.427 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:01.441 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:01.441 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:01.439+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:01.458 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:01.475 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:01.475 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:01.532 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65783 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 153600~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902762973999998, 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049043995999999999, 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65783 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:10:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627919+0000", 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627921+0000", 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9599999999999999e-06 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627927+0000", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627932+0000", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.994e-06 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676768+0000", 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048836308000000002 2024-09-18T01:10:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676814+0000", 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6017999999999999e-05 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676963+0000", 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014852999999999999 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65784 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 176128~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902737989000002, 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049301365999999999, 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65784 2024-09-18T01:10:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627944+0000", 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627946+0000", 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1100000000000001e-06 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627953+0000", 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4910000000000002e-06 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627958+0000", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9400000000000001e-06 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.676986+0000", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049027617000000003 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677044+0000", 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8925000000000002e-05 2024-09-18T01:10:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677245+0000", 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000200712 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65785 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 198656~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902718210000003, 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049671932000000002, 2024-09-18T01:10:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65785 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627964+0000", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627970+0000", 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2009999999999996e-06 2024-09-18T01:10:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627977+0000", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7519999999999999e-06 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627982+0000", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677293+0000", 2024-09-18T01:10:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049311290000000001 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677376+0000", 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2798000000000002e-05 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677635+0000", 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025934999999999999 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65786 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 221184~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902691808, 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050040446000000002, 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65786 2024-09-18T01:10:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627990+0000", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.627993+0000", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1289999999999999e-06 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628000+0000", 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2420000000000003e-06 2024-09-18T01:10:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628005+0000", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677696+0000", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.049691708000000001 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.677783+0000", 2024-09-18T01:10:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6847000000000004e-05 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678030+0000", 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024721900000000002 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65787 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 243712~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:10:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902670311999998, 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050551287, 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65787 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628011+0000", 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628012+0000", 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628014+0000", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628021+0000", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9909999999999996e-06 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628025+0000", 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7779999999999998e-06 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678107+0000", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050081254999999998 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678320+0000", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000213303 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678563+0000", 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000242735 2024-09-18T01:10:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65788 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 266240~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902651015999993, 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050846257999999998, 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65788 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628031+0000", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628033+0000", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.841e-06 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628039+0000", 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9769999999999997e-06 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628044+0000", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.651e-06 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678583+0000", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050539321999999998 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678626+0000", 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2639999999999998e-05 2024-09-18T01:10:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678877+0000", 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025149999999999999 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65789 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 288768~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902633182000002, 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051204804999999999, 2024-09-18T01:10:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65789 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628049+0000", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628051+0000", 2024-09-18T01:10:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.844e-06 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628057+0000", 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628069+0000", 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2523e-05 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.678978+0000", 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.050908762000000003 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679059+0000", 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1137999999999995e-05 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679253+0000", 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019437 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65790 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 311296~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902607118999995, 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051751599000000002, 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65790 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628075+0000", 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628076+0000", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.401e-06 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628078+0000", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1380000000000001e-06 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628084+0000", 2024-09-18T01:10:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1330000000000004e-06 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628089+0000", 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7140000000000001e-06 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679273+0000", 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051184260000000002 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679414+0000", 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014070099999999999 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679826+0000", 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000412252 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65791 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 333824~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902587764000003, 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052278968000000002, 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65791 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628094+0000", 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628095+0000", 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628099+0000", 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628106+0000", 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8959999999999997e-06 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628118+0000", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1389999999999999e-05 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.679845+0000", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.051727283999999998 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680151+0000", 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000306495 2024-09-18T01:10:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680373+0000", 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221499 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65792 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 356352~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902557520000002, 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052499974999999997, 2024-09-18T01:10:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65792 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628124+0000", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628125+0000", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628127+0000", 2024-09-18T01:10:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628133+0000", 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1399999999999997e-06 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628138+0000", 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6750000000000001e-06 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680394+0000", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052255929 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680447+0000", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3477999999999999e-05 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680624+0000", 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017717699999999999 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65793 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 378880~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902538925000002, 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052777690000000002, 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65793 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628143+0000", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628145+0000", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628151+0000", 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628155+0000", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5639999999999998e-06 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680665+0000", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052509183000000001 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680719+0000", 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4329000000000001e-05 2024-09-18T01:10:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680921+0000", 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000201552 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65794 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 401408~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902520366000005, 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053093195000000003, 2024-09-18T01:10:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65794 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628161+0000", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628162+0000", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2500000000000001e-07 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628164+0000", 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1900000000000002e-06 2024-09-18T01:10:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628170+0000", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8499999999999999e-06 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628174+0000", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6190000000000002e-06 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.680956+0000", 2024-09-18T01:10:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052781533999999998 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681037+0000", 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0890000000000006e-05 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681255+0000", 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021778699999999999 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65795 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 423936~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902502311999996, 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053347797000000002, 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65795 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628179+0000", 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628180+0000", 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628182+0000", 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.846e-06 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628187+0000", 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8039999999999998e-06 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628192+0000", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6009999999999997e-06 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681293+0000", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053100533999999998 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681345+0000", 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2129000000000002e-05 2024-09-18T01:10:01.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681527+0000", 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000182552 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65796 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 446464~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902484552999994, 2024-09-18T01:10:01.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053657009999999998, 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65796 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628197+0000", 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628198+0000", 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5700000000000002e-07 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628205+0000", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.99e-06 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628212+0000", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3520000000000001e-06 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628217+0000", 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681558+0000", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053341274000000001 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681638+0000", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9867e-05 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681854+0000", 2024-09-18T01:10:01.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021602700000000001 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65797 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 468992~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902458284000005, 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053895131999999998, 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65797 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628223+0000", 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628224+0000", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7700000000000005e-07 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628226+0000", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.193e-06 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628232+0000", 2024-09-18T01:10:01.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628237+0000", 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681891+0000", 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053653816999999999 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.681938+0000", 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682119+0000", 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018081400000000001 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65798 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 491520~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902439663999999, 2024-09-18T01:10:01.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054198983999999999, 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65798 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628242+0000", 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628243+0000", 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628244+0000", 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7749999999999999e-06 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628250+0000", 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.806e-06 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628255+0000", 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682148+0000", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053893625000000001 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682224+0000", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5949000000000001e-05 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682441+0000", 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021666400000000001 2024-09-18T01:10:01.605 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65799 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:34 [read 514048~2048 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.902421414000003, 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054453278000000001, 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.606 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65799 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628260+0000", 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628261+0000", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628263+0000", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9470000000000002e-06 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628269+0000", 2024-09-18T01:10:01.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628273+0000", 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1849999999999997e-06 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682477+0000", 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054204535999999998 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682523+0000", 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5327999999999997e-05 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682714+0000", 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019093199999999999 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65905 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.610 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.901870039000002, 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.054092841000000003, 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65905 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628812+0000", 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5400000000000002e-07 2024-09-18T01:10:01.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628814+0000", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-06 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628820+0000", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0159999999999997e-06 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.628825+0000", 2024-09-18T01:10:01.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682755+0000", 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.053930064999999999 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682828+0000", 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2860000000000004e-05 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682905+0000", 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6536e-05 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65915 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.900271052999997, 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052621526000000002, 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.615 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65915 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630411+0000", 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630412+0000", 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.077e-06 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630417+0000", 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3539999999999999e-06 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630425+0000", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0339999999999997e-06 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.630430+0000", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5870000000000003e-06 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682940+0000", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.052510687 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.682984+0000", 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3078000000000003e-05 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683032+0000", 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8708999999999997e-05 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:65922 3.18 3:18caab08:::rbd_data.117658cbc921.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:10:01.619 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.896505066000003, 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048973397000000002, 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 65922 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:01.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634177+0000", 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634178+0000", 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.316e-06 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634184+0000", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6910000000000002e-06 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634194+0000", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0659000000000001e-05 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.634211+0000", 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6688e-05 2024-09-18T01:10:01.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683051+0000", 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.048839585999999997 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683104+0000", 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3223999999999997e-05 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:01.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:08:53.683150+0000", 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6233000000000003e-05 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:01.624 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:01.624 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:01.530+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:01.625 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:01.669 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:01.668+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:01.704 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-18T01:10:01.732 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:01.732 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:01.732 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:01.732 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:01.733 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:01.732+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:01.741 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:01.834 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:01.833+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:01.885 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:01.885 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:01.885 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:01.885 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:01.885 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:01.893 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:01.935 INFO:teuthology.orchestra.run.smithi120.stdout:3770 read 0x31e49f5 thru 0x31f3336 (0xe942 bytes) 2024-09-18T01:10:01.936 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:01.934+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:01.936 INFO:teuthology.orchestra.run.smithi120.stdout:3771 read 0x365ecb5 thru 0x366d8e2 (0xec2e bytes) 2024-09-18T01:10:01.937 INFO:teuthology.orchestra.run.smithi120.stdout:3772 punch from 0x172f57d to 0x17373da, (0x7e5d bytes) 2024-09-18T01:10:01.941 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:01.941 INFO:teuthology.orchestra.run.smithi120.stdout:3775 read 0x16b63b3 thru 0x16c42af (0xdefd bytes) 2024-09-18T01:10:01.942 INFO:teuthology.orchestra.run.smithi120.stdout:3776 punch from 0x566ba2d to 0x566e1b8, (0x278b bytes) 2024-09-18T01:10:01.946 INFO:teuthology.orchestra.run.smithi120.stdout:3778 trunc from 0xa76b3e5 to 0xd4bc58d 2024-09-18T01:10:01.951 INFO:teuthology.orchestra.run.smithi120.stdout:3779 punch from 0x747ca4 to 0x74ab18, (0x2e74 bytes) 2024-09-18T01:10:01.951 INFO:teuthology.orchestra.run.smithi120.stdout:3780 write 0xc39b445 thru 0xc3a0816 (0x53d2 bytes) 2024-09-18T01:10:01.972 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:01.972 INFO:teuthology.orchestra.run.smithi120.stdout:3782 punch from 0x26d83c0 to 0x26e6d7c, (0xe9bc bytes) 2024-09-18T01:10:01.972 INFO:teuthology.orchestra.run.smithi120.stdout:3783 punch from 0xb0f15cc to 0xb0fee13, (0xd847 bytes) 2024-09-18T01:10:01.972 INFO:teuthology.orchestra.run.smithi120.stdout:3784 write 0x2639b5a thru 0x2646b2c (0xcfd3 bytes) 2024-09-18T01:10:01.979 INFO:teuthology.orchestra.run.smithi120.stdout:3785 read 0xaf9fbca thru 0xafae5aa (0xe9e1 bytes) 2024-09-18T01:10:01.979 INFO:teuthology.orchestra.run.smithi120.stdout:3787 write 0xcf805ab thru 0xcf80c1b (0x671 bytes) 2024-09-18T01:10:01.985 INFO:teuthology.orchestra.run.smithi120.stdout:3788 write 0x685ab83 thru 0x6863a37 (0x8eb5 bytes) 2024-09-18T01:10:02.003 INFO:teuthology.orchestra.run.smithi120.stdout:3790 read 0xc7a9974 thru 0xc7af163 (0x57f0 bytes) 2024-09-18T01:10:02.003 INFO:teuthology.orchestra.run.smithi120.stdout:3791 trunc from 0xd4bc58d to 0xdb9ea18 2024-09-18T01:10:02.009 INFO:teuthology.orchestra.run.smithi120.stdout:3792 punch from 0x39c399b to 0x39c7638, (0x3c9d bytes) 2024-09-18T01:10:02.009 INFO:teuthology.orchestra.run.smithi120.stdout:3793 punch from 0x4997e88 to 0x49a7417, (0xf58f bytes) 2024-09-18T01:10:02.009 INFO:teuthology.orchestra.run.smithi120.stdout:3797 write 0x61b7f29 thru 0x61c1464 (0x953c bytes) 2024-09-18T01:10:02.026 INFO:teuthology.orchestra.run.smithi120.stdout:3799 read 0x598268b thru 0x59857b9 (0x312f bytes) 2024-09-18T01:10:02.027 INFO:teuthology.orchestra.run.smithi120.stdout:3800 punch from 0x56c025d to 0x56cc9f0, (0xc793 bytes) 2024-09-18T01:10:02.027 INFO:teuthology.orchestra.run.smithi120.stdout:3801 write 0xdb6c3a8 thru 0xdb7b0bb (0xed14 bytes) 2024-09-18T01:10:02.036 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:02.035+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:02.046 INFO:teuthology.orchestra.run.smithi120.stdout:3804 trunc from 0xdb9ea18 to 0xe0e9c1a 2024-09-18T01:10:02.051 INFO:teuthology.orchestra.run.smithi120.stdout:3806 punch from 0x712c219 to 0x7132a9a, (0x6881 bytes) 2024-09-18T01:10:02.051 INFO:teuthology.orchestra.run.smithi120.stdout:3808 punch from 0x2bd5551 to 0x2be437f, (0xee2e bytes) 2024-09-18T01:10:02.051 INFO:teuthology.orchestra.run.smithi120.stdout:3810 read 0xc37c049 thru 0xc380199 (0x4151 bytes) 2024-09-18T01:10:02.052 INFO:teuthology.orchestra.run.smithi120.stdout:3811 read 0x7ccd7c0 thru 0x7cdc1af (0xe9f0 bytes) 2024-09-18T01:10:02.052 INFO:teuthology.orchestra.run.smithi120.stdout:3812 read 0xae3873e thru 0xae418f5 (0x91b8 bytes) 2024-09-18T01:10:02.052 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:02.060 INFO:teuthology.orchestra.run.smithi120.stdout:3815 trunc from 0xe0e9c1a to 0x73bb8cf 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 124.336275324, 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3052488020000004, 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3052392240000001, 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3052251979999996, 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:02.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:02.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3051964250000001, 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:02.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:02.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.305168729, 2024-09-18T01:10:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:02.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:02.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:02.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:02.096 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.305134281, 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:02.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:02.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3051044349999996, 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:02.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3050822440000003, 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:02.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:02.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3050453380000002, 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:02.112 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:02.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:02.117 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3050242230000002, 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:02.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3049936979999996, 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:02.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:02.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2995134440000005, 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:02.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2994595850000001, 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:02.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2993821629999998, 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:02.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2993520490000003, 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2993021310000001, 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:02.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2992685440000002, 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:02.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:02.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:02.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2992440099999998, 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:02.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:02.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2990587280000003, 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.169 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:02.174 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:02.169+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3816 trunc from 0x73bb8cf to 0x9b17617 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3818 trunc from 0x9b17617 to 0xc11709f 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3820 read 0x7cc4373 thru 0x7ccc163 (0x7df1 bytes) 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3821 read 0x4dc556a thru 0x4dd1b45 (0xc5dc bytes) 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3824 write 0xe4e24ed thru 0xe4f0ebb (0xe9cf bytes) 2024-09-18T01:10:02.174 INFO:teuthology.orchestra.run.smithi120.stdout:3825 trunc from 0xe4f0ebc to 0x1260adb 2024-09-18T01:10:02.269 INFO:teuthology.orchestra.run.smithi120.stdout:3826 read 0xed96c2 thru 0xee77e7 (0xe126 bytes) 2024-09-18T01:10:02.270 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:02.269+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:02.271 INFO:teuthology.orchestra.run.smithi120.stdout:3827 read 0x318d77 thru 0x319e2f (0x10b9 bytes) 2024-09-18T01:10:02.271 INFO:teuthology.orchestra.run.smithi120.stdout:3828 clone 36 order 22 su 65536 sc 3 2024-09-18T01:10:02.325 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:02.325 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:02.325 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.333 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:02.377 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:02.374+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:02.473 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:02.472+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:02.478 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:02.479 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:02.574 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:02.573+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:02.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:10:02.637 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0363639569999998, 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:02.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:02.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0352139420000004, 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:02.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0351946310000004, 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.651 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0351668849999998, 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0351470330000003, 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.660 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0351271689999999, 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:02.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:02.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0351089629999999, 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:02.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0350889160000003, 2024-09-18T01:10:02.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:02.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0350707080000001, 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:02.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:02.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0350525959999999, 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:02.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:02.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0350346779999997, 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.035016658, 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:02.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:02.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.034463702, 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:02.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0344449789999999, 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:02.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:02.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0344151960000003, 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:02.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0343511080000001, 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:02.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:02.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0341668070000001, 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.715 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0341486529999999, 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:02.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0341202809999999, 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:02.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0340983550000002, 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:02.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:02.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:02.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:02.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:02.729 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:02.731 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:02.728+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:02.831 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:02.830+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:02.833 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:02.834 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:02.932 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:02.931+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:02.987 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:02.987 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:02.987 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:02.988 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:02.988 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:02.996 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:03.033 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:03.032+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:03.134 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:03.133+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:03.140 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.567915120999999, 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:03.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:03.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:03.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:03.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.515923978000004, 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:03.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:03.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.515847182000002, 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:03.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:03.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.511342029000005, 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:03.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.162 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.511086233, 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.163 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:03.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:03.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.507060905000003, 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:03.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.506919754999998, 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:03.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.506686850999998, 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:03.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.504067789000004, 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.184 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:03.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:03.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:03.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:03.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 93.481728755000006, 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:03.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531213480999995, 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:03.195 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:03.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:03.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:03.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531191739999997, 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531171332, 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:03.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:03.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531151672999997, 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:03.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531118337999999, 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:03.216 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:03.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:03.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531083805999998, 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:03.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531038916, 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:03.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:03.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.531012540999996, 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:03.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.530993390000006, 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:03.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:03.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 69.530974822000005, 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:03.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.248 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:03.249 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:03.250 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:03.248+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:03.350 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:03.349+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:03.394 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:03.394 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:03.394 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:03.394 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:03.402 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:03.451 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:03.450+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:03.486 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:03.501 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:03.501 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:03.502 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-18T01:10:03.502 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:10:03.502 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:10:03.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-18T01:10:03.545 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:03.545 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:03.545 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:03.545 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:03.545 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:03.554 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:03.574 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:03.572+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:10:03.636 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:03.637 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:03.652 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:03.651+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:03.665 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:03.665 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "" 2024-09-18T01:10:03.665 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:03.701 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:03.701 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 171.800463651, 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:03.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:03.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:03.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.076535281000005, 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.0765131, 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:03.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:03.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.109008633000002, 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:03.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:03.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.1040363040000001, 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:03.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0991439209999996, 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:03.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:03.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0937292830000001, 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:03.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0821550169999998, 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:03.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:03.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.745 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0731873240000001, 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:03.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:03.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.750 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0731328619999996, 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:03.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0678614880000001, 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:03.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:03.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0677807489999998, 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:03.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:03.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.766 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0668197499999996, 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0667609579999997, 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:03.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:03.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0655969900000004, 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:03.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0652323209999999, 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:03.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:03.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:03.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0647097849999998, 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:03.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:03.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0643124039999998, 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:03.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.797 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0624596740000003, 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:03.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:03.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 6.0623374830000003, 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:03.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:03.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:03.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:03.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:03.808 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:03.809 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:03.810 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:03.810 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:03.811 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:03.811 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:03.811 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:03.812 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:03.812 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:03.812 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:03.813 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:03.807+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:03.813 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:03.813 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:03.832 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:03.860 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:03.860 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:03.910 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:03.909+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:03.924 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:03.924 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:03.933 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:04.011 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:04.010+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:04.035 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.035 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.044 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:04.111 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:04.110+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:04.146 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.146 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.155 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:04.212 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:04.211+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:04.258 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.258 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:04.313 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:04.311+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:04.369 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.369 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:04.414 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:04.413+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:04.515 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:04.513+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:04.521 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.521 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.530 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:04.616 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:04.614+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:04.654 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-18T01:10:04.682 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.683 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.694 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:04.735 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:04.733+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:04.818 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:04.816+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:04.844 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.844 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:04.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:04.883 INFO:teuthology.orchestra.run.smithi120.stdout:3829 writesame 0x4ada7bd thru 0x4ae75c6 data_size 0x6(0xce0a bytes) 2024-09-18T01:10:04.886 INFO:teuthology.orchestra.run.smithi120.stdout:3832 trunc from 0x4ae75c7 to 0x6513aa3 2024-09-18T01:10:04.891 INFO:teuthology.orchestra.run.smithi120.stdout:3833 write 0x3416a83 thru 0x3422c97 (0xc215 bytes) 2024-09-18T01:10:04.906 INFO:teuthology.orchestra.run.smithi120.stdout:3834 write 0x860c08d thru 0x860d87c (0x17f0 bytes) 2024-09-18T01:10:04.910 INFO:teuthology.orchestra.run.smithi120.stdout:3836 punch from 0x3c6e78b to 0x3c729d5, (0x424a bytes) 2024-09-18T01:10:04.916 INFO:teuthology.orchestra.run.smithi120.stdout:3837 trunc from 0x860d87d to 0x34e0c72 2024-09-18T01:10:04.919 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:04.918+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:04.928 INFO:teuthology.orchestra.run.smithi120.stdout:3838 write 0x1e18cb8 thru 0x1e1b325 (0x266e bytes) 2024-09-18T01:10:04.931 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:04.942 INFO:teuthology.orchestra.run.smithi120.stdout:3842 writesame 0x76d61d6 thru 0x76df688 data_size 0x1(0x94b3 bytes) 2024-09-18T01:10:04.942 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:04.942 INFO:teuthology.orchestra.run.smithi120.stdout:3848 punch from 0x37dc6d5 to 0x37e9375, (0xcca0 bytes) 2024-09-18T01:10:04.956 INFO:teuthology.orchestra.run.smithi120.stdout:3849 write 0xe19181a thru 0xe193337 (0x1b1e bytes) 2024-09-18T01:10:04.959 INFO:teuthology.orchestra.run.smithi120.stdout:3850 write 0x1377be4 thru 0x1385d5d (0xe17a bytes) 2024-09-18T01:10:04.963 INFO:teuthology.orchestra.run.smithi120.stdout:3851 punch from 0xc8e256b to 0xc8e3068, (0xafd bytes) 2024-09-18T01:10:04.978 INFO:teuthology.orchestra.run.smithi120.stdout:3852 trunc from 0xe193338 to 0xf6e09a 2024-09-18T01:10:04.999 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:04.999 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.008 INFO:teuthology.orchestra.run.smithi120.stdout:3853 write 0x971ca44 thru 0x9721fcb (0x5588 bytes) 2024-09-18T01:10:05.011 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:05.020 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:05.019+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:05.024 INFO:teuthology.orchestra.run.smithi120.stdout:3854 trunc from 0x9721fcc to 0x641032 2024-09-18T01:10:05.044 INFO:teuthology.orchestra.run.smithi120.stdout:3855 punch from 0x50fc81 to 0x510e6b, (0x11ea bytes) 2024-09-18T01:10:05.044 INFO:teuthology.orchestra.run.smithi120.stdout:3856 punch from 0x15952c to 0x15e338, (0x4e0c bytes) 2024-09-18T01:10:05.044 INFO:teuthology.orchestra.run.smithi120.stdout:3858 punch from 0x2dc2df to 0x2e171d, (0x543e bytes) 2024-09-18T01:10:05.055 INFO:teuthology.orchestra.run.smithi120.stdout:3859 write 0x5272173 thru 0x527d038 (0xaec6 bytes) 2024-09-18T01:10:05.075 INFO:teuthology.orchestra.run.smithi120.stdout:3860 writesame 0xd66d783 thru 0xd6735ac data_size 0x11(0x5e2a bytes) 2024-09-18T01:10:05.075 INFO:teuthology.orchestra.run.smithi120.stdout:3861 writesame 0x4d08ecb thru 0x4d14c64 data_size 0x7(0xbd9a bytes) 2024-09-18T01:10:05.080 INFO:teuthology.orchestra.run.smithi120.stdout:3863 write 0x16fc708 thru 0x16fd570 (0xe69 bytes) 2024-09-18T01:10:05.084 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.084 INFO:teuthology.orchestra.run.smithi120.stdout:3865 punch from 0x4760528 to 0x476efb3, (0xea8b bytes) 2024-09-18T01:10:05.084 INFO:teuthology.orchestra.run.smithi120.stdout:3867 punch from 0x23faa06 to 0x2400c1b, (0x6215 bytes) 2024-09-18T01:10:05.084 INFO:teuthology.orchestra.run.smithi120.stdout:3868 read 0xd255cf4 thru 0xd263b05 (0xde12 bytes) 2024-09-18T01:10:05.085 INFO:teuthology.orchestra.run.smithi120.stdout:3869 punch from 0x463be72 to 0x4644a07, (0x8b95 bytes) 2024-09-18T01:10:05.085 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.085 INFO:teuthology.orchestra.run.smithi120.stdout:3872 write 0xc67b659 thru 0xc684ab7 (0x945f bytes) 2024-09-18T01:10:05.091 INFO:teuthology.orchestra.run.smithi120.stdout:3873 read 0x447711f thru 0x447d37d (0x625f bytes) 2024-09-18T01:10:05.091 INFO:teuthology.orchestra.run.smithi120.stdout:3874 writesame 0xa886951 thru 0xa89454f data_size 0x1(0xdbff bytes) 2024-09-18T01:10:05.121 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:05.120+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:05.127 INFO:teuthology.orchestra.run.smithi120.stdout:3875 write 0xdbeb00c thru 0xdbef696 (0x468b bytes) 2024-09-18T01:10:05.130 INFO:teuthology.orchestra.run.smithi120.stdout:3876 write 0x82359fb thru 0x823d3c7 (0x79cd bytes) 2024-09-18T01:10:05.133 INFO:teuthology.orchestra.run.smithi120.stdout:3878 write 0xafe1e27 thru 0xafe997a (0x7b54 bytes) 2024-09-18T01:10:05.137 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.137 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.141 INFO:teuthology.orchestra.run.smithi120.stdout:3880 trunc from 0xdbef697 to 0x7d61bbd 2024-09-18T01:10:05.150 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:05.156 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.156 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.156 INFO:teuthology.orchestra.run.smithi120.stdout:3886 read 0x58b4a3f thru 0x58bdfe2 (0x95a4 bytes) 2024-09-18T01:10:05.168 INFO:teuthology.orchestra.run.smithi120.stdout:3887 write 0xcc663b2 thru 0xcc75d42 (0xf991 bytes) 2024-09-18T01:10:05.173 INFO:teuthology.orchestra.run.smithi120.stdout:3888 punch from 0xb287a8e to 0xb2952d3, (0xd845 bytes) 2024-09-18T01:10:05.173 INFO:teuthology.orchestra.run.smithi120.stdout:3889 writesame 0xc3efcdf thru 0xc3f8797 data_size 0x1(0x8ab9 bytes) 2024-09-18T01:10:05.193 INFO:teuthology.orchestra.run.smithi120.stdout:3890 write 0x58fa278 thru 0x59023ce (0x8157 bytes) 2024-09-18T01:10:05.197 INFO:teuthology.orchestra.run.smithi120.stdout:3891 read 0x8b6267f thru 0x8b66efc (0x487e bytes) 2024-09-18T01:10:05.197 INFO:teuthology.orchestra.run.smithi120.stdout:3892 trunc from 0xcc75d43 to 0xcc82388 2024-09-18T01:10:05.202 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.204 INFO:teuthology.orchestra.run.smithi120.stdout:3896 trunc from 0xcc82388 to 0xb4fd842 2024-09-18T01:10:05.216 INFO:teuthology.orchestra.run.smithi120.stdout:3897 writesame 0x7a0d9b6 thru 0x7a172e1 data_size 0x2(0x992c bytes) 2024-09-18T01:10:05.222 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:05.221+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:3901 read 0x15c11d thru 0x16706b (0xaf4f bytes) 2024-09-18T01:10:05.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.228 INFO:teuthology.orchestra.run.smithi120.stdout:3903 read 0x7aa37a6 thru 0x7aac8ac (0x9107 bytes) 2024-09-18T01:10:05.228 INFO:teuthology.orchestra.run.smithi120.stdout:3904 read 0x5e9f2c7 thru 0x5e9fecc (0xc06 bytes) 2024-09-18T01:10:05.229 INFO:teuthology.orchestra.run.smithi120.stdout:3905 writesame 0x79eaf4a thru 0x79f5879 data_size 0x8(0xa930 bytes) 2024-09-18T01:10:05.246 INFO:teuthology.orchestra.run.smithi120.stdout:3906 trunc from 0xb4fd842 to 0x2206d9f 2024-09-18T01:10:05.263 INFO:teuthology.orchestra.run.smithi120.stdout:3907 read 0x7ab2c7 thru 0x7af290 (0x3fca bytes) 2024-09-18T01:10:05.270 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.270 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.273 INFO:teuthology.orchestra.run.smithi120.stdout:3909 writesame 0x613ba52 thru 0x613f20b data_size 0x7(0x37ba bytes) 2024-09-18T01:10:05.273 INFO:teuthology.orchestra.run.smithi120.stdout:3910 read 0x1d4aadd thru 0x1d5598b (0xaeaf bytes) 2024-09-18T01:10:05.281 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:05.289 INFO:teuthology.orchestra.run.smithi120.stdout:3911 write 0xdde6e55 thru 0xddeb9bf (0x4b6b bytes) 2024-09-18T01:10:05.292 INFO:teuthology.orchestra.run.smithi120.stdout:3912 punch from 0xd7a95af to 0xd7abb28, (0x2579 bytes) 2024-09-18T01:10:05.293 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.293 INFO:teuthology.orchestra.run.smithi120.stdout:3918 write 0x7ccef37 thru 0x7cd1123 (0x21ed bytes) 2024-09-18T01:10:05.296 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.299 INFO:teuthology.orchestra.run.smithi120.stdout:3920 trunc from 0xddeb9c0 to 0xb252aac 2024-09-18T01:10:05.309 INFO:teuthology.orchestra.run.smithi120.stdout:3922 read 0x7d3553d thru 0x7d3699d (0x1461 bytes) 2024-09-18T01:10:05.313 INFO:teuthology.orchestra.run.smithi120.stdout:3923 trunc from 0xb252aac to 0x7c3fd0f 2024-09-18T01:10:05.321 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.322 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:05.321+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:05.325 INFO:teuthology.orchestra.run.smithi120.stdout:3927 trunc from 0x7c3fd0f to 0x4e0f155 2024-09-18T01:10:05.334 INFO:teuthology.orchestra.run.smithi120.stdout:3928 punch from 0x23ca94b to 0x23cb94e, (0x1003 bytes) 2024-09-18T01:10:05.334 INFO:teuthology.orchestra.run.smithi120.stdout:3929 writesame 0xd6f88 thru 0xde33f data_size 0x1c(0x73b8 bytes) 2024-09-18T01:10:05.344 INFO:teuthology.orchestra.run.smithi120.stdout:3930 trunc from 0x4e0f155 to 0xb2933ec 2024-09-18T01:10:05.350 INFO:teuthology.orchestra.run.smithi120.stdout:3931 read 0x3c1bdba thru 0x3c2061e (0x4865 bytes) 2024-09-18T01:10:05.350 INFO:teuthology.orchestra.run.smithi120.stdout:3932 read 0x74d315c thru 0x74d3580 (0x425 bytes) 2024-09-18T01:10:05.350 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.350 INFO:teuthology.orchestra.run.smithi120.stdout:3935 read 0x406eec7 thru 0x40701ba (0x12f4 bytes) 2024-09-18T01:10:05.351 INFO:teuthology.orchestra.run.smithi120.stdout:3937 trunc from 0xb2933ec to 0xa3bea58 2024-09-18T01:10:05.366 INFO:teuthology.orchestra.run.smithi120.stdout:3938 trunc from 0xa3bea58 to 0x39a763a 2024-09-18T01:10:05.380 INFO:teuthology.orchestra.run.smithi120.stdout:3941 trunc from 0x39a763a to 0x8a36606 2024-09-18T01:10:05.384 INFO:teuthology.orchestra.run.smithi120.stdout:3943 write 0x7c8b25b thru 0x7c8fdf9 (0x4b9f bytes) 2024-09-18T01:10:05.388 INFO:teuthology.orchestra.run.smithi120.stdout:3945 trunc from 0x8a36606 to 0x8309ae7 2024-09-18T01:10:05.398 INFO:teuthology.orchestra.run.smithi120.stdout:3947 trunc from 0x8309ae7 to 0xaefbdd3 2024-09-18T01:10:05.410 INFO:teuthology.orchestra.run.smithi120.stdout:3948 trunc from 0xaefbdd3 to 0xec890c5 2024-09-18T01:10:05.416 INFO:teuthology.orchestra.run.smithi120.stdout:3949 read 0xc4706b8 thru 0xc479c59 (0x95a2 bytes) 2024-09-18T01:10:05.416 INFO:teuthology.orchestra.run.smithi120.stdout:3950 read 0x36a207a thru 0x36b057b (0xe502 bytes) 2024-09-18T01:10:05.416 INFO:teuthology.orchestra.run.smithi120.stdout:3951 writesame 0x88c1b7c thru 0x88d0368 data_size 0x3(0xe7ed bytes) 2024-09-18T01:10:05.420 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.420 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.423 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:05.422+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:05.429 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:3952 read 0x14fe43e thru 0x1506226 (0x7de9 bytes) 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:3953 read 0x7a39bd6 thru 0x7a3e91d (0x4d48 bytes) 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:3955 punch from 0x3f7bc75 to 0x3f83d31, (0x80bc bytes) 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:3957 read 0x62fcf65 thru 0x6308d52 (0xbdee bytes) 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:3958 read 0xcf1ee90 thru 0xcf2e939 (0xfaaa bytes) 2024-09-18T01:10:05.430 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.445 INFO:teuthology.orchestra.run.smithi120.stdout:3960 trunc from 0xec890c5 to 0x1765a19 2024-09-18T01:10:05.473 INFO:teuthology.orchestra.run.smithi120.stdout:3961 write 0x8fd77db thru 0x8fe17a5 (0x9fcb bytes) 2024-09-18T01:10:05.492 INFO:teuthology.orchestra.run.smithi120.stdout:3962 write 0xea89126 thru 0xea94396 (0xb271 bytes) 2024-09-18T01:10:05.496 INFO:teuthology.orchestra.run.smithi120.stdout:3963 write 0xe50cf9f thru 0xe5113cf (0x4431 bytes) 2024-09-18T01:10:05.499 INFO:teuthology.orchestra.run.smithi120.stdout:3964 writesame 0x602aa3c thru 0x602d97a data_size 0x5(0x2f3f bytes) 2024-09-18T01:10:05.506 INFO:teuthology.orchestra.run.smithi120.stdout:3965 write 0x8052270 thru 0x805b010 (0x8da1 bytes) 2024-09-18T01:10:05.509 INFO:teuthology.orchestra.run.smithi120.stdout:3966 punch from 0xb464df8 to 0xb46a9fd, (0x5c05 bytes) 2024-09-18T01:10:05.509 INFO:teuthology.orchestra.run.smithi120.stdout:3967 punch from 0x5af8f0 to 0x5bc1b9, (0xc8c9 bytes) 2024-09-18T01:10:05.509 INFO:teuthology.orchestra.run.smithi120.stdout:3969 writesame 0x8ae3415 thru 0x8aee82e data_size 0x1(0xb41a bytes) 2024-09-18T01:10:05.524 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:05.523+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:05.530 INFO:teuthology.orchestra.run.smithi120.stdout:3970 writesame 0x8d383c thru 0x8d5935 data_size 0x15(0x20fa bytes) 2024-09-18T01:10:05.531 INFO:teuthology.orchestra.run.smithi120.stdout:3971 writesame 0xe5bf041 thru 0xe5c794f data_size 0x1(0x890f bytes) 2024-09-18T01:10:05.540 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.540 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:05.557 INFO:teuthology.orchestra.run.smithi120.stdout:3972 trunc from 0xea94397 to 0x6a00fdc 2024-09-18T01:10:05.575 INFO:teuthology.orchestra.run.smithi120.stdout:3973 trunc from 0x6a00fdc to 0x97dd75d 2024-09-18T01:10:05.579 INFO:teuthology.orchestra.run.smithi120.stdout:3975 read 0x1ecde62 thru 0x1ed2bf9 (0x4d98 bytes) 2024-09-18T01:10:05.590 INFO:teuthology.orchestra.run.smithi120.stdout:3976 write 0xca79fc5 thru 0xca7ba36 (0x1a72 bytes) 2024-09-18T01:10:05.593 INFO:teuthology.orchestra.run.smithi120.stdout:3977 read 0x9be241c thru 0x9be7455 (0x503a bytes) 2024-09-18T01:10:05.593 INFO:teuthology.orchestra.run.smithi120.stdout:3979 write 0xaa51e27 thru 0xaa5388b (0x1a65 bytes) 2024-09-18T01:10:05.596 INFO:teuthology.orchestra.run.smithi120.stdout:3980 writesame 0x384fa60 thru 0x385a4f4 data_size 0x1(0xaa95 bytes) 2024-09-18T01:10:05.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:10:05.620 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.630 INFO:teuthology.orchestra.run.smithi120.stdout:3982 trunc from 0xca7ba37 to 0x39a4c58 2024-09-18T01:10:05.631 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:05.630+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:05.644 INFO:teuthology.orchestra.run.smithi120.stdout:3984 write 0x20ff7fa thru 0x210da6d (0xe274 bytes) 2024-09-18T01:10:05.648 INFO:teuthology.orchestra.run.smithi120.stdout:3985 write 0xee287f thru 0xef2665 (0xfde7 bytes) 2024-09-18T01:10:05.651 INFO:teuthology.orchestra.run.smithi120.stdout:3986 punch from 0x360cbe5 to 0x3612a73, (0x5e8e bytes) 2024-09-18T01:10:05.660 INFO:teuthology.orchestra.run.smithi120.stdout:3987 write 0x658a60e thru 0x659944f (0xee42 bytes) 2024-09-18T01:10:05.664 INFO:teuthology.orchestra.run.smithi120.stdout:3988 read 0x210d397 thru 0x211a688 (0xd2f2 bytes) 2024-09-18T01:10:05.665 INFO:teuthology.orchestra.run.smithi120.stdout:3991 write 0x54d5602 thru 0x54d9510 (0x3f0f bytes) 2024-09-18T01:10:05.673 INFO:teuthology.orchestra.run.smithi120.stdout:3992 trunc from 0x6599450 to 0xa516cbb 2024-09-18T01:10:05.678 INFO:teuthology.orchestra.run.smithi120.stdout:3993 read 0x6d2e2e5 thru 0x6d33613 (0x532f bytes) 2024-09-18T01:10:05.684 INFO:teuthology.orchestra.run.smithi120.stdout:3995 trunc from 0xa516cbb to 0xea11af2 2024-09-18T01:10:05.699 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.699 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:3996 trunc from 0xea11af2 to 0x202bf02 2024-09-18T01:10:05.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:05.721 INFO:teuthology.orchestra.run.smithi120.stdout:3997 trunc from 0x202bf02 to 0x32b7f6b 2024-09-18T01:10:05.726 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:05.725+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:05.727 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.745 INFO:teuthology.orchestra.run.smithi120.stdout:3999 writesame 0xca2544b thru 0xca2e82f data_size 0x1(0x93e5 bytes) 2024-09-18T01:10:05.745 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.745 INFO:teuthology.orchestra.run.smithi120.stdout:4001 write 0xb641518 thru 0xb646dd4 (0x58bd bytes) 2024-09-18T01:10:05.761 INFO:teuthology.orchestra.run.smithi120.stdout:4002 trunc from 0xca2e830 to 0x3316f9f 2024-09-18T01:10:05.771 INFO:teuthology.orchestra.run.smithi120.stdout:4003 read 0xa6d98b thru 0xa760d3 (0x8749 bytes) 2024-09-18T01:10:05.787 INFO:teuthology.orchestra.run.smithi120.stdout:4004 write 0xa434bd7 thru 0xa43a6c4 (0x5aee bytes) 2024-09-18T01:10:05.790 INFO:teuthology.orchestra.run.smithi120.stdout:4005 read 0x1e57896 thru 0x1e59957 (0x20c2 bytes) 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:4006 punch from 0x89d1c52 to 0x89d5bf8, (0x3fa6 bytes) 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:4009 read 0x6811c73 thru 0x681fa7e (0xde0c bytes) 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:4010 punch from 0xa08d35 to 0xa0e307, (0x55d2 bytes) 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:4011 read 0x42ceda7 thru 0x42ddd4c (0xefa6 bytes) 2024-09-18T01:10:05.791 INFO:teuthology.orchestra.run.smithi120.stdout:4012 write 0x427765a thru 0x427acc2 (0x3669 bytes) 2024-09-18T01:10:05.794 INFO:teuthology.orchestra.run.smithi120.stdout:4013 punch from 0x39e5821 to 0x39f07ae, (0xaf8d bytes) 2024-09-18T01:10:05.794 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.794 INFO:teuthology.orchestra.run.smithi120.stdout:4015 read 0x961e236 thru 0x96288c1 (0xa68c bytes) 2024-09-18T01:10:05.794 INFO:teuthology.orchestra.run.smithi120.stdout:4016 writesame 0x3c3a440 thru 0x3c3d055 data_size 0x1(0x2c16 bytes) 2024-09-18T01:10:05.802 INFO:teuthology.orchestra.run.smithi120.stdout:4017 punch from 0x16315d0 to 0x163f1f1, (0xdc21 bytes) 2024-09-18T01:10:05.807 INFO:teuthology.orchestra.run.smithi120.stdout:4018 trunc from 0xa43a6c5 to 0x5a14f73 2024-09-18T01:10:05.818 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.818 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.823 INFO:teuthology.orchestra.run.smithi120.stdout:4019 trunc from 0x5a14f73 to 0x138c60e 2024-09-18T01:10:05.826 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:05.825+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:05.827 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:05.838 INFO:teuthology.orchestra.run.smithi120.stdout:4020 read 0x614676 thru 0x616806 (0x2191 bytes) 2024-09-18T01:10:05.839 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.850 INFO:teuthology.orchestra.run.smithi120.stdout:4022 write 0x5f0d6e8 thru 0x5f0db3c (0x455 bytes) 2024-09-18T01:10:05.854 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.854 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.860 INFO:teuthology.orchestra.run.smithi120.stdout:4025 write 0x5ffcca1 thru 0x6003c1a (0x6f7a bytes) 2024-09-18T01:10:05.865 INFO:teuthology.orchestra.run.smithi120.stdout:4027 read 0xa68ab2 thru 0xa71064 (0x85b3 bytes) 2024-09-18T01:10:05.866 INFO:teuthology.orchestra.run.smithi120.stdout:4029 write 0x37762f0 thru 0x3784e16 (0xeb27 bytes) 2024-09-18T01:10:05.871 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.871 INFO:teuthology.orchestra.run.smithi120.stdout:4031 read 0xc8e352 thru 0xc9703e (0x8ced bytes) 2024-09-18T01:10:05.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:05.885 INFO:teuthology.orchestra.run.smithi120.stdout:4032 write 0xbf31c90 thru 0xbf33a0b (0x1d7c bytes) 2024-09-18T01:10:05.889 INFO:teuthology.orchestra.run.smithi120.stdout:4033 writesame 0x7ca06d3 thru 0x7ca1e94 data_size 0x1(0x17c2 bytes) 2024-09-18T01:10:05.904 INFO:teuthology.orchestra.run.smithi120.stdout:4034 write 0xe467d83 thru 0xe4773da (0xf658 bytes) 2024-09-18T01:10:05.910 INFO:teuthology.orchestra.run.smithi120.stdout:4037 read 0x5ca9875 thru 0x5cb47de (0xaf6a bytes) 2024-09-18T01:10:05.911 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.911 INFO:teuthology.orchestra.run.smithi120.stdout:4039 write 0xc86bf9d thru 0xc8782cb (0xc32f bytes) 2024-09-18T01:10:05.917 INFO:teuthology.orchestra.run.smithi120.stdout:4040 write 0xb104eea thru 0xb10b967 (0x6a7e bytes) 2024-09-18T01:10:05.928 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:05.927+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:05.933 INFO:teuthology.orchestra.run.smithi120.stdout:4041 trunc from 0xe4773db to 0x31b276f 2024-09-18T01:10:05.947 INFO:teuthology.orchestra.run.smithi120.stdout:4042 punch from 0x2867000 to 0x2870b82, (0x9b82 bytes) 2024-09-18T01:10:05.947 INFO:teuthology.orchestra.run.smithi120.stdout:4043 write 0x19a52a4 thru 0x19ab109 (0x5e66 bytes) 2024-09-18T01:10:05.967 INFO:teuthology.orchestra.run.smithi120.stdout:4044 writesame 0xc805efe thru 0xc811178 data_size 0x1(0xb27b bytes) 2024-09-18T01:10:05.967 INFO:teuthology.orchestra.run.smithi120.stdout:4047 write 0xa5c0dbe thru 0xa5ceb77 (0xddba bytes) 2024-09-18T01:10:05.971 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.971 INFO:teuthology.orchestra.run.smithi120.stdout:4049 punch from 0x798baaa to 0x799b95a, (0xfeb0 bytes) 2024-09-18T01:10:05.971 INFO:teuthology.orchestra.run.smithi120.stdout:4050 punch from 0x5d293dc to 0x5d38e81, (0xfaa5 bytes) 2024-09-18T01:10:05.971 INFO:teuthology.orchestra.run.smithi120.stdout:4051 write 0x6201a3 thru 0x626dc9 (0x6c27 bytes) 2024-09-18T01:10:05.972 INFO:teuthology.orchestra.run.smithi120.stdout:4053 read 0x42d11a0 thru 0x42ddb2f (0xc990 bytes) 2024-09-18T01:10:05.973 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:05.973 INFO:teuthology.orchestra.run.smithi120.stdout:4055 punch from 0xb5251c2 to 0xb52729e, (0x20dc bytes) 2024-09-18T01:10:05.973 INFO:teuthology.orchestra.run.smithi120.stdout:4056 writesame 0x72fcd3a thru 0x72fe12a data_size 0x1(0x13f1 bytes) 2024-09-18T01:10:05.977 INFO:teuthology.orchestra.run.smithi120.stdout:4057 read 0x2159428 thru 0x216930a (0xfee3 bytes) 2024-09-18T01:10:05.978 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:05.978 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:05.978 INFO:teuthology.orchestra.run.smithi120.stdout:4058 write 0x21816f1 thru 0x2188975 (0x7285 bytes) 2024-09-18T01:10:05.986 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:05.989 INFO:teuthology.orchestra.run.smithi120.stdout:4059 trunc from 0xc811179 to 0x4151291 2024-09-18T01:10:05.999 INFO:teuthology.orchestra.run.smithi120.stdout:4060 write 0x30684f9 thru 0x306e138 (0x5c40 bytes) 2024-09-18T01:10:06.004 INFO:teuthology.orchestra.run.smithi120.stdout:4061 trunc from 0x4151291 to 0x6592c9e 2024-09-18T01:10:06.011 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:06.011 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:06.024 INFO:teuthology.orchestra.run.smithi120.stdout:4062 write 0xce54da0 thru 0xce59b62 (0x4dc3 bytes) 2024-09-18T01:10:06.028 INFO:teuthology.orchestra.run.smithi120.stdout:4064 writesame 0x4e25de2 thru 0x4e2be8a data_size 0x31(0x60a9 bytes) 2024-09-18T01:10:06.030 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:06.030 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:06.030 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:06.029+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:06.031 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.036 INFO:teuthology.orchestra.run.smithi120.stdout:4066 trunc from 0xce59b63 to 0x958eb1e 2024-09-18T01:10:06.047 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:06.047 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:06.050 INFO:teuthology.orchestra.run.smithi120.stdout:4068 trunc from 0x958eb1e to 0x5f4d3a5 2024-09-18T01:10:06.064 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:06.065 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:06.065 INFO:teuthology.orchestra.run.smithi120.stdout:4070 write 0x72ed8e8 thru 0x72f8fd5 (0xb6ee bytes) 2024-09-18T01:10:06.082 INFO:teuthology.orchestra.run.smithi120.stdout:4071 writesame 0xb5ad01a thru 0xb5b0b85 data_size 0x2(0x3b6c bytes) 2024-09-18T01:10:06.082 INFO:teuthology.orchestra.run.smithi120.stdout:4072 punch from 0x6af6a15 to 0x6af8f4a, (0x2535 bytes) 2024-09-18T01:10:06.082 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.088 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:06.088 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:06.091 INFO:teuthology.orchestra.run.smithi120.stdout:4074 write 0xcf57b40 thru 0xcf58f7e (0x143f bytes) 2024-09-18T01:10:06.094 INFO:teuthology.orchestra.run.smithi120.stdout:4075 read 0x26ed9da thru 0x26f3627 (0x5c4e bytes) 2024-09-18T01:10:06.094 INFO:teuthology.orchestra.run.smithi120.stdout:4076 read 0x2afda71 thru 0x2b03cfb (0x628b bytes) 2024-09-18T01:10:06.094 INFO:teuthology.orchestra.run.smithi120.stdout:4078 punch from 0x6961bbc to 0x6965716, (0x3b5a bytes) 2024-09-18T01:10:06.096 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:06.096 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:06.105 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:06.106 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:06.106 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:06.109 INFO:teuthology.orchestra.run.smithi120.stdout:4082 trunc from 0xcf58f7f to 0x21588e 2024-09-18T01:10:06.126 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:06.126 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:06.129 INFO:teuthology.orchestra.run.smithi120.stdout:4083 write 0x9cd42f thru 0x9d033f (0x2f11 bytes) 2024-09-18T01:10:06.129 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:06.128+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:06.130 INFO:teuthology.orchestra.run.smithi120.stdout:4084 punch from 0x248c02 to 0x257a48, (0xee46 bytes) 2024-09-18T01:10:06.141 INFO:teuthology.orchestra.run.smithi120.stdout:4085 write 0x5247c41 thru 0x52508e1 (0x8ca1 bytes) 2024-09-18T01:10:06.144 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:06.144 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:06.147 INFO:teuthology.orchestra.run.smithi120.stdout:4086 write 0x25ee38d thru 0x25f37b4 (0x5428 bytes) 2024-09-18T01:10:06.154 INFO:teuthology.orchestra.run.smithi120.stdout:4087 read 0x23be968 thru 0x23c8b8b (0xa224 bytes) 2024-09-18T01:10:06.154 INFO:teuthology.orchestra.run.smithi120.stdout:4088 read 0x4cd2700 thru 0x4cdab5c (0x845d bytes) 2024-09-18T01:10:06.154 INFO:teuthology.orchestra.run.smithi120.stdout:4090 writesame 0x2979d7 thru 0x29f698 data_size 0x3(0x7cc2 bytes) 2024-09-18T01:10:06.171 INFO:teuthology.orchestra.run.smithi120.stdout:4091 writesame 0x878ea4b thru 0x8798227 data_size 0x1(0x97dd bytes) 2024-09-18T01:10:06.181 INFO:teuthology.orchestra.run.smithi120.stdout:4092 write 0xb94c651 thru 0xb94f9de (0x338e bytes) 2024-09-18T01:10:06.184 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.184 INFO:teuthology.orchestra.run.smithi120.stdout:4094 read 0x6028c11 thru 0x602d77c (0x4b6c bytes) 2024-09-18T01:10:06.184 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.184 INFO:teuthology.orchestra.run.smithi120.stdout:4096 punch from 0x6bf39d2 to 0x6bf5d07, (0x2335 bytes) 2024-09-18T01:10:06.185 INFO:teuthology.orchestra.run.smithi120.stdout:4097 trunc from 0xb94f9df to 0xad01a9a 2024-09-18T01:10:06.195 INFO:teuthology.orchestra.run.smithi120.stdout:4098 write 0x97a5981 thru 0x97aa7b7 (0x4e37 bytes) 2024-09-18T01:10:06.198 INFO:teuthology.orchestra.run.smithi120.stdout:4099 read 0x8499ab8 thru 0x84a0936 (0x6e7f bytes) 2024-09-18T01:10:06.198 INFO:teuthology.orchestra.run.smithi120.stdout:4101 read 0x25fda10 thru 0x2606698 (0x8c89 bytes) 2024-09-18T01:10:06.199 INFO:teuthology.orchestra.run.smithi120.stdout:4102 read 0x1d823a7 thru 0x1d8920d (0x6e67 bytes) 2024-09-18T01:10:06.199 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.205 INFO:teuthology.orchestra.run.smithi120.stdout:4104 writesame 0xb29ba56 thru 0xb2a92b6 data_size 0x1(0xd861 bytes) 2024-09-18T01:10:06.205 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.206 INFO:teuthology.orchestra.run.smithi120.stdout:4106 trunc from 0xb2a92b7 to 0xb1accd9 2024-09-18T01:10:06.211 INFO:teuthology.orchestra.run.smithi120.stdout:4107 punch from 0x6449820 to 0x6458824, (0xf004 bytes) 2024-09-18T01:10:06.211 INFO:teuthology.orchestra.run.smithi120.stdout:4108 writesame 0x99c59b9 thru 0x99d0209 data_size 0x35(0xa851 bytes) 2024-09-18T01:10:06.215 INFO:teuthology.orchestra.run.smithi120.stdout:4112 writesame 0x2ba32fd thru 0x2ba4e23 data_size 0x7(0x1b27 bytes) 2024-09-18T01:10:06.218 INFO:teuthology.orchestra.run.smithi120.stdout:4115 writesame 0x27c4e91 thru 0x27cfe6a data_size 0x52(0xafda bytes) 2024-09-18T01:10:06.220 INFO:teuthology.orchestra.run.smithi120.stdout:4116 write 0x7bf58f5 thru 0x7bfaa81 (0x518d bytes) 2024-09-18T01:10:06.230 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:06.229+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:06.231 INFO:teuthology.orchestra.run.smithi120.stdout:4117 write 0xcc9f3f2 thru 0xcca3cfd (0x490c bytes) 2024-09-18T01:10:06.242 INFO:teuthology.orchestra.run.smithi120.stdout:4118 write 0xd6b96b1 thru 0xd6bd1b9 (0x3b09 bytes) 2024-09-18T01:10:06.243 INFO:teuthology.orchestra.run.smithi120.stdout:4119 writesame 0x82ee770 thru 0x82f4727 data_size 0xc(0x5fb8 bytes) 2024-09-18T01:10:06.248 INFO:teuthology.orchestra.run.smithi120.stdout:4120 write 0x988c0f1 thru 0x988f8cb (0x37db bytes) 2024-09-18T01:10:06.249 INFO:teuthology.orchestra.run.smithi120.stdout:4121 read 0x5186e50 thru 0x5196108 (0xf2b9 bytes) 2024-09-18T01:10:06.250 INFO:teuthology.orchestra.run.smithi120.stdout:4122 write 0x5fb7dba thru 0x5fb81bb (0x402 bytes) 2024-09-18T01:10:06.253 INFO:teuthology.orchestra.run.smithi120.stdout:4123 writesame 0x95cf8ce thru 0x95d50a3 data_size 0x1(0x57d6 bytes) 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6539359769999997, 2024-09-18T01:10:06.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:06.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:06.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6527859620000003, 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:06.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6527666510000003, 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:06.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:06.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6527389049999996, 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:06.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:06.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:06.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:06.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6527190530000002, 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:06.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:06.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6526991889999998, 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:06.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:06.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6526809829999998, 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:06.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6526609360000002, 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:06.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:06.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.652642728, 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:06.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.305 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6526246159999998, 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6526066979999996, 2024-09-18T01:10:06.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:06.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:06.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6525886779999999, 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:06.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:06.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6520357219999999, 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:06.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6520169989999998, 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:06.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6519872160000002, 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.331 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:06.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.651923128, 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:06.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:06.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:06.340 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.651738827, 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:06.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6517206729999998, 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.345 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:06.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.350 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6516923009999998, 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.6516703750000001, 2024-09-18T01:10:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.360 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:06.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:06.362 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:06.360+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4124 read 0x58560ca thru 0x5858aa9 (0x29e0 bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4125 trunc from 0xd6bd1ba to 0xa2726aa 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4126 writesame 0x29fbd thru 0x2c4bd data_size 0x1(0x2501 bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4128 trunc from 0xa2726aa to 0x91e32b1 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4129 write 0x9e97ac2 thru 0x9e9eef7 (0x7436 bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4131 writesame 0xca8aac8 thru 0xca8bf33 data_size 0x2(0x146c bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4134 read 0x7a16197 thru 0x7a193cd (0x3237 bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4135 write 0x50d7515 thru 0x50e4391 (0xce7d bytes) 2024-09-18T01:10:06.364 INFO:teuthology.orchestra.run.smithi120.stdout:4136 write 0x2e7b395 thru 0x2e88df4 (0xda60 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4137 punch from 0xbcc22f0 to 0xbccc8ba, (0xa5ca bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4138 writesame 0x3e8707b thru 0x3e9428c data_size 0x3(0xd212 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4139 write 0xa73419c thru 0xa736c5c (0x2ac1 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4140 writesame 0xd4f444b thru 0xd4f62a7 data_size 0x1(0x1e5d bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4141 punch from 0xb20c22c to 0xb21622c, (0xa000 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4142 read 0xceae123 thru 0xcebd9e0 (0xf8be bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4143 write 0xb0ba5fd thru 0xb0c77ad (0xd1b1 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4144 write 0x36d489 thru 0x3775a8 (0xa120 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4145 read 0xf30386 thru 0xf3e8ae (0xe529 bytes) 2024-09-18T01:10:06.365 INFO:teuthology.orchestra.run.smithi120.stdout:4147 trunc from 0xd4f62a8 to 0x502b8a5 2024-09-18T01:10:06.379 INFO:teuthology.orchestra.run.smithi120.stdout:4148 punch from 0xda0a7d to 0xda3313, (0x2896 bytes) 2024-09-18T01:10:06.385 INFO:teuthology.orchestra.run.smithi120.stdout:4149 writesame 0x570a7db thru 0x571156f data_size 0x9(0x6d95 bytes) 2024-09-18T01:10:06.397 INFO:teuthology.orchestra.run.smithi120.stdout:4150 writesame 0xa06bd1f thru 0xa078695 data_size 0x5(0xc977 bytes) 2024-09-18T01:10:06.398 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.407 INFO:teuthology.orchestra.run.smithi120.stdout:4152 trunc from 0xa078696 to 0x15c268c 2024-09-18T01:10:06.424 INFO:teuthology.orchestra.run.smithi120.stdout:4153 read 0xe8078f thru 0xe8c809 (0xc07b bytes) 2024-09-18T01:10:06.424 INFO:teuthology.orchestra.run.smithi120.stdout:4154 punch from 0x514b65 to 0x52326e, (0xe709 bytes) 2024-09-18T01:10:06.437 INFO:teuthology.orchestra.run.smithi120.stdout:4155 trunc from 0x15c268c to 0xced4b0c 2024-09-18T01:10:06.444 INFO:teuthology.orchestra.run.smithi120.stdout:4156 read 0x7bac4f8 thru 0x7bb6c83 (0xa78c bytes) 2024-09-18T01:10:06.444 INFO:teuthology.orchestra.run.smithi120.stdout:4157 read 0xc7eac0d thru 0xc7ed6f1 (0x2ae5 bytes) 2024-09-18T01:10:06.444 INFO:teuthology.orchestra.run.smithi120.stdout:4159 read 0xc0b8bd9 thru 0xc0c87c5 (0xfbed bytes) 2024-09-18T01:10:06.444 INFO:teuthology.orchestra.run.smithi120.stdout:4160 write 0x9e68d4b thru 0x9e777e2 (0xea98 bytes) 2024-09-18T01:10:06.458 INFO:teuthology.orchestra.run.smithi120.stdout:4163 writesame 0xd8585f1 thru 0xd860723 data_size 0x31(0x8133 bytes) 2024-09-18T01:10:06.458 INFO:teuthology.orchestra.run.smithi120.stdout:4164 writesame 0x993ba98 thru 0x9949a99 data_size 0x6a(0xe002 bytes) 2024-09-18T01:10:06.462 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:06.461+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout:4166 read 0x131f097 thru 0x131fc79 (0xbe3 bytes) 2024-09-18T01:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout:4169 writesame 0xc2dbdfe thru 0xc2df463 data_size 0x21(0x3666 bytes) 2024-09-18T01:10:06.468 INFO:teuthology.orchestra.run.smithi120.stdout:4170 writesame 0x4c7253d thru 0x4c7f53f data_size 0x1(0xd003 bytes) 2024-09-18T01:10:06.512 INFO:teuthology.orchestra.run.smithi120.stdout:4171 trunc from 0xd860724 to 0xfd8342 2024-09-18T01:10:06.515 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:06.515 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:228693 3.a 3:510697df:::rbd_data.117692a25d3b.0000000000000014:head [delete] snapc 4b=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:10:06.514406+0000", 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00085588700000000005, 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00088561399999999995, 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 228693 2024-09-18T01:10:06.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514406+0000", 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514406+0000", 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514407+0000", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514418+0000", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0373e-05 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514435+0000", 2024-09-18T01:10:06.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7105e-05 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514445+0000", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0202e-05 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514513+0000", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7779000000000003e-05 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514618+0000", 2024-09-18T01:10:06.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010532800000000001 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:10:06.514748+0000", 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012937499999999999 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-18T01:10:06.520 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:06.525 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:06.533 INFO:teuthology.orchestra.run.smithi120.stdout:4172 write 0x3e7d9f2 thru 0x3e854c4 (0x7ad3 bytes) 2024-09-18T01:10:06.545 INFO:teuthology.orchestra.run.smithi120.stdout:4173 writesame 0x64ca694 thru 0x64ce732 data_size 0x1(0x409f bytes) 2024-09-18T01:10:06.545 INFO:teuthology.orchestra.run.smithi120.stdout:4175 read 0x6455fb1 thru 0x6462963 (0xc9b3 bytes) 2024-09-18T01:10:06.545 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.556 INFO:teuthology.orchestra.run.smithi120.stdout:4177 writesame 0x84b1a56 thru 0x84b3c1d data_size 0x8(0x21c8 bytes) 2024-09-18T01:10:06.556 INFO:teuthology.orchestra.run.smithi120.stdout:4178 write 0xabb749 thru 0xac266a (0x6f22 bytes) 2024-09-18T01:10:06.563 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:06.562+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:06.566 INFO:teuthology.orchestra.run.smithi120.stdout:4179 write 0xa109552 thru 0xa1157d7 (0xc286 bytes) 2024-09-18T01:10:06.572 INFO:teuthology.orchestra.run.smithi120.stdout:4180 write 0x2d6d8f thru 0x2e4d3d (0xdfaf bytes) 2024-09-18T01:10:06.573 INFO:teuthology.orchestra.run.smithi120.stdout:4183 writesame 0x3cf4703 thru 0x3d00b32 data_size 0x49(0xc430 bytes) 2024-09-18T01:10:06.583 INFO:teuthology.orchestra.run.smithi120.stdout:4185 trunc from 0xa1157d8 to 0x4349694 2024-09-18T01:10:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:10:06.612 INFO:teuthology.orchestra.run.smithi120.stdout:4186 write 0xb2968b3 thru 0xb29f284 (0x89d2 bytes) 2024-09-18T01:10:06.616 INFO:teuthology.orchestra.run.smithi120.stdout:4187 write 0x5875c76 thru 0x58793d5 (0x3760 bytes) 2024-09-18T01:10:06.620 INFO:teuthology.orchestra.run.smithi120.stdout:4188 write 0x8a29f39 thru 0x8a3676f (0xc837 bytes) 2024-09-18T01:10:06.635 INFO:teuthology.orchestra.run.smithi120.stdout:4191 write 0xdbafaeb thru 0xdbbd1d9 (0xd6ef bytes) 2024-09-18T01:10:06.641 INFO:teuthology.orchestra.run.smithi120.stdout:4192 write 0x849f900 thru 0x84a3e62 (0x4563 bytes) 2024-09-18T01:10:06.647 INFO:teuthology.orchestra.run.smithi120.stdout:4193 read 0x4a26e59 thru 0x4a31573 (0xa71b bytes) 2024-09-18T01:10:06.647 INFO:teuthology.orchestra.run.smithi120.stdout:4194 read 0x3d55e18 thru 0x3d63cd6 (0xdebf bytes) 2024-09-18T01:10:06.648 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.649 INFO:teuthology.orchestra.run.smithi120.stdout:4197 read 0x83f11a4 thru 0x83f7a33 (0x6890 bytes) 2024-09-18T01:10:06.649 INFO:teuthology.orchestra.run.smithi120.stdout:4199 read 0x759eeb9 thru 0x75a5faa (0x70f2 bytes) 2024-09-18T01:10:06.649 INFO:teuthology.orchestra.run.smithi120.stdout:4200 write 0x9701cd3 thru 0x970ef0b (0xd239 bytes) 2024-09-18T01:10:06.651 INFO:teuthology.orchestra.run.smithi120.stdout:4201 punch from 0xa24f8fe to 0xa2514ab, (0x1bad bytes) 2024-09-18T01:10:06.652 INFO:teuthology.orchestra.run.smithi120.stdout:4203 read 0x16f9284 thru 0x1702d20 (0x9a9d bytes) 2024-09-18T01:10:06.652 INFO:teuthology.orchestra.run.smithi120.stdout:4204 writesame 0x8b88bdb thru 0x8b8ecf1 data_size 0x5(0x6117 bytes) 2024-09-18T01:10:06.656 INFO:teuthology.orchestra.run.smithi120.stdout:4205 read 0x2923aef thru 0x292ed0c (0xb21e bytes) 2024-09-18T01:10:06.663 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:06.662+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:06.671 INFO:teuthology.orchestra.run.smithi120.stdout:4206 trunc from 0xdbbd1da to 0x1592d8c 2024-09-18T01:10:06.677 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:06.677 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:06.677 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:06.686 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:06.690 INFO:teuthology.orchestra.run.smithi120.stdout:4207 trunc from 0x1592d8c to 0x7040d5f 2024-09-18T01:10:06.696 INFO:teuthology.orchestra.run.smithi120.stdout:4208 read 0x4d77199 thru 0x4d7ce03 (0x5c6b bytes) 2024-09-18T01:10:06.696 INFO:teuthology.orchestra.run.smithi120.stdout:4209 write 0x6541a48 thru 0x655105e (0xf617 bytes) 2024-09-18T01:10:06.702 INFO:teuthology.orchestra.run.smithi120.stdout:4210 punch from 0x4de82c6 to 0x4deefd4, (0x6d0e bytes) 2024-09-18T01:10:06.704 INFO:teuthology.orchestra.run.smithi120.stdout:4211 trunc from 0x7040d5f to 0x4e4abe4 2024-09-18T01:10:06.715 INFO:teuthology.orchestra.run.smithi120.stdout:4212 writesame 0x2686ce1 thru 0x2693113 data_size 0x1(0xc433 bytes) 2024-09-18T01:10:06.737 INFO:teuthology.orchestra.run.smithi120.stdout:4214 write 0x7bbdfa thru 0x7bfc3f (0x3e46 bytes) 2024-09-18T01:10:06.738 INFO:teuthology.orchestra.run.smithi120.stdout:4216 read 0x46a0f8b thru 0x46a4047 (0x30bd bytes) 2024-09-18T01:10:06.746 INFO:teuthology.orchestra.run.smithi120.stdout:4217 write 0x5f96f10 thru 0x5f97370 (0x461 bytes) 2024-09-18T01:10:06.760 INFO:teuthology.orchestra.run.smithi120.stdout:4218 writesame 0x7f4a823 thru 0x7f503bf data_size 0x1(0x5b9d bytes) 2024-09-18T01:10:06.760 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.760 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.761 INFO:teuthology.orchestra.run.smithi120.stdout:4222 read 0x220896 thru 0x225c4f (0x53ba bytes) 2024-09-18T01:10:06.761 INFO:teuthology.orchestra.run.smithi120.stdout:4223 write 0x59d11e0 thru 0x59d13b4 (0x1d5 bytes) 2024-09-18T01:10:06.764 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:06.763+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:06.773 INFO:teuthology.orchestra.run.smithi120.stdout:4225 write 0xb02b420 thru 0xb02f7ce (0x43af bytes) 2024-09-18T01:10:06.776 INFO:teuthology.orchestra.run.smithi120.stdout:4226 punch from 0xc20389 to 0xc2e46d, (0xe0e4 bytes) 2024-09-18T01:10:06.776 INFO:teuthology.orchestra.run.smithi120.stdout:4227 write 0x2e72917 thru 0x2e7d917 (0xb001 bytes) 2024-09-18T01:10:06.777 INFO:teuthology.orchestra.run.smithi120.stdout:4229 punch from 0xa86628e to 0xa86a472, (0x41e4 bytes) 2024-09-18T01:10:06.786 INFO:teuthology.orchestra.run.smithi120.stdout:4230 trunc from 0xb02f7cf to 0x3ee4339 2024-09-18T01:10:06.798 INFO:teuthology.orchestra.run.smithi120.stdout:4231 read 0x2909174 thru 0x2910f1c (0x7da9 bytes) 2024-09-18T01:10:06.798 INFO:teuthology.orchestra.run.smithi120.stdout:4232 read 0x28c2ed3 thru 0x28c881d (0x594b bytes) 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.225439508999997, 2024-09-18T01:10:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.173448366000002, 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.17337157, 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.168866417000004, 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.168610620999999, 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.164585293000002, 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.164444142999997, 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.164211238999997, 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.161592177000003, 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.139253143000005, 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188737868999993, 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188716127999996, 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188695719999998, 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188676060999995, 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188642725999998, 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188608193999997, 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188563303999999, 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188536928999994, 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188517778000005, 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:06.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:06.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.188499210000003, 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:06.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:06.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:06.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:06.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:06.908 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:06.910 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:06.911 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:06.908+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:06.912 INFO:teuthology.orchestra.run.smithi120.stdout:4234 writesame 0x81d6fbb thru 0x81dabbd data_size 0x9(0x3c03 bytes) 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4235 punch from 0x7a53ff7 to 0x7a60804, (0xc80d bytes) 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4236 trunc from 0x81dabbe to 0x7404c11 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4237 write 0xb90b9dc thru 0xb9165db (0xac00 bytes) 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4238 writesame 0x9272e36 thru 0x9280b24 data_size 0x3(0xdcef bytes) 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4239 write 0x395066a thru 0x395ff70 (0xf907 bytes) 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4241 trunc from 0xb9165dc to 0x9b32e1b 2024-09-18T01:10:06.913 INFO:teuthology.orchestra.run.smithi120.stdout:4242 punch from 0x5046f90 to 0x505471f, (0xd78f bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4243 punch from 0x4897d1f to 0x489ba83, (0x3d64 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4248 punch from 0x8c2c26c to 0x8c35c18, (0x99ac bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4249 punch from 0x1684ed to 0x16c9c1, (0x44d4 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4250 read 0x628edc9 thru 0x6299247 (0xa47f bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4252 write 0x3891045 thru 0x389d49c (0xc458 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4256 write 0xc4fe473 thru 0xc507e08 (0x9996 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4258 write 0x3e1561b thru 0x3e15d68 (0x74e bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4259 read 0xd5a4f8 thru 0xd61930 (0x7439 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4260 trunc from 0xc507e09 to 0x9155940 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4261 read 0x63354ee thru 0x6341135 (0xbc48 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4262 punch from 0x3505531 to 0x350a94a, (0x5419 bytes) 2024-09-18T01:10:06.914 INFO:teuthology.orchestra.run.smithi120.stdout:4263 trunc from 0x9155940 to 0x73af18 2024-09-18T01:10:06.943 INFO:teuthology.orchestra.run.smithi120.stdout:4264 write 0x9c430fb thru 0x9c4c8f1 (0x97f7 bytes) 2024-09-18T01:10:06.947 INFO:teuthology.orchestra.run.smithi120.stdout:4265 read 0x11175c2 thru 0x11207d3 (0x9212 bytes) 2024-09-18T01:10:06.953 INFO:teuthology.orchestra.run.smithi120.stdout:4267 trunc from 0x9c4c8f2 to 0x56e0246 2024-09-18T01:10:06.965 INFO:teuthology.orchestra.run.smithi120.stdout:4268 write 0x3fa8cf5 thru 0x3fb18a5 (0x8bb1 bytes) 2024-09-18T01:10:06.971 INFO:teuthology.orchestra.run.smithi120.stdout:4269 read 0x671e4c thru 0x67ae27 (0x8fdc bytes) 2024-09-18T01:10:06.971 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.986 INFO:teuthology.orchestra.run.smithi120.stdout:4271 writesame 0xca832b7 thru 0xca8bbef data_size 0x1(0x8939 bytes) 2024-09-18T01:10:06.987 INFO:teuthology.orchestra.run.smithi120.stdout:4272 read 0x7856de6 thru 0x78573e6 (0x601 bytes) 2024-09-18T01:10:06.987 INFO:teuthology.orchestra.run.smithi120.stdout:4273 read 0x4b42629 thru 0x4b4b77a (0x9152 bytes) 2024-09-18T01:10:06.993 INFO:teuthology.orchestra.run.smithi120.stdout:4274 writesame 0xe4c3b26 thru 0xe4caefd data_size 0x2c(0x73d8 bytes) 2024-09-18T01:10:06.994 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:06.994 INFO:teuthology.orchestra.run.smithi120.stdout:4277 read 0x277836f thru 0x27867d4 (0xe466 bytes) 2024-09-18T01:10:06.994 INFO:teuthology.orchestra.run.smithi120.stdout:4278 write 0x6b17ba1 thru 0x6b1e4d0 (0x6930 bytes) 2024-09-18T01:10:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:4279 punch from 0x39c6622 to 0x39d4e7d, (0xe85b bytes) 2024-09-18T01:10:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:4280 punch from 0x1fc218 to 0x203df1, (0x7bd9 bytes) 2024-09-18T01:10:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:4282 read 0x8337c00 thru 0x83471ad (0xf5ae bytes) 2024-09-18T01:10:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:4283 read 0x6497dba thru 0x649ab67 (0x2dae bytes) 2024-09-18T01:10:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:4285 write 0x92660e7 thru 0x9270a21 (0xa93b bytes) 2024-09-18T01:10:07.002 INFO:teuthology.orchestra.run.smithi120.stdout:4286 clone 37 order 19 su 16384 sc 14 2024-09-18T01:10:07.010 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:07.009+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:07.058 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:07.058 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:07.058 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:07.058 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:07.066 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:07.111 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:07.109+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:07.211 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:07.211 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:07.211 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:07.212 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:07.210+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:07.219 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:07.312 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:07.311+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:07.322 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:07.322 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 175.42298351400001, 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:07.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:07.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:07.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:07.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:07.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.699055143999999, 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:07.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:07.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.699032962999993, 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.731528496000003, 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:07.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.726556167, 2024-09-18T01:10:07.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:07.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:07.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7216637840000004, 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.351 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7162491459999991, 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:07.356 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:07.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:07.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:07.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7046748800000007, 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:07.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.366 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.695707187, 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.367 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:07.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6956527250000004, 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.372 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:07.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:07.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:07.377 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6903813509999992, 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.378 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:07.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6903006119999997, 2024-09-18T01:10:07.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:07.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:07.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:07.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6893396129999996, 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:07.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:07.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:07.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:07.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6892808210000005, 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6881168530000004, 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:07.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.409 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6877521840000007, 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.410 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:07.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:07.416 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6872296480000006, 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6868322669999998, 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.422 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:07.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:07.427 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6849795370000002, 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:07.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:07.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:07.432 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.6848573459999994, 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:07.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:07.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:07.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:07.438 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:07.440 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:07.438+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:07.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:07.540 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:07.539+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:07.547 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:07.548 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:07.548 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:07.548 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:07.556 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:07.642 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:07.641+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:07.703 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:07.704 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:07.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:07.742 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:07.741+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:07.843 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:07.842+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 130.13290316699999, 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:07.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:07.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:07.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.109445077, 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.096366825, 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:07.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:07.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.096346568, 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:07.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:07.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.096001533000001, 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:07.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095694403, 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:07.887 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:07.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:07.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:07.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095658644, 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.894 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:07.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:07.899 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095620374999999, 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.905 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095595903, 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:07.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095575242000001, 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:07.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095556618, 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.918 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:07.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:07.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:07.923 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095536827, 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:07.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095517724, 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:07.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:07.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:07.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095499633999999, 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095481534999999, 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.939 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:07.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095463206, 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:07.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:07.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095444840000001, 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:07.950 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.951 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:07.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:07.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.09542658, 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:07.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:07.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.095408126000001, 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:07.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.09535844, 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:07.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:07.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:07.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:07.976 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:07.977 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:07.978 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:07.976+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:08.077 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:08.076+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:08.121 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:08.121 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:08.121 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:08.121 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:08.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:08.156 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:08.214 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:08.213+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:08.277 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:08.278 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:08.278 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:08.279 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:08.278+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:08.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:08.293 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:08.310 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:08.326 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:08.342 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:08.359 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:08.376 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:08.379 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:08.378+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:08.393 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:08.393 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:08.398 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:08.398 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:08.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:08.410 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:08.410 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:08.480 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:08.479+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:08.512 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:08.512 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:08.522 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:08.581 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:08.580+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:08.623 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:10:08.628 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:08.628 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:08.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:08.680 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:08.680 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:08.681 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-18T01:10:08.681 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:10:08.681 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:10:08.681 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-18T01:10:08.682 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:08.681+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:08.784 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:08.783+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:08.819 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:08.819 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:08.829 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:08.849 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:08.849 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:10:08.850 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:08.900 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:08.899+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:08.941 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:08.941 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:08.950 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:08.987 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:08.986+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:09.054 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.054 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:09.107 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:09.106+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:09.165 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.165 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:09.217 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:09.216+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:09.278 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.278 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.287 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:09.288 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:09.287+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:09.389 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:09.388+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:09.430 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.430 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:09.490 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:09.489+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:09.542 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.542 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:10:09.622 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:09.621+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:09.692 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:09.691+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:09.696 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.696 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.706 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:09.727 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-18T01:10:09.783 INFO:teuthology.orchestra.run.smithi120.stdout:4287 trunc from 0xe4caefe to 0x1ba6b15 2024-09-18T01:10:09.793 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:09.792+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:09.824 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.824 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.834 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:09.842 INFO:teuthology.orchestra.run.smithi120.stdout:4288 write 0xdbc48e1 thru 0xdbd1ad7 (0xd1f7 bytes) 2024-09-18T01:10:09.852 INFO:teuthology.orchestra.run.smithi120.stdout:4289 writesame 0x635cb89 thru 0x6365485 data_size 0x1(0x88fd bytes) 2024-09-18T01:10:09.865 INFO:teuthology.orchestra.run.smithi120.stdout:4292 read 0x3b66166 thru 0x3b69868 (0x3703 bytes) 2024-09-18T01:10:09.865 INFO:teuthology.orchestra.run.smithi120.stdout:4293 punch from 0x76f7f0f to 0x7705ab1, (0xdba2 bytes) 2024-09-18T01:10:09.866 INFO:teuthology.orchestra.run.smithi120.stdout:4294 read 0x158cedd thru 0x15945a5 (0x76c9 bytes) 2024-09-18T01:10:09.866 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:09.866 INFO:teuthology.orchestra.run.smithi120.stdout:4298 writesame 0x9d02c33 thru 0x9d0ca9e data_size 0x2(0x9e6c bytes) 2024-09-18T01:10:09.886 INFO:teuthology.orchestra.run.smithi120.stdout:4299 writesame 0xe36c1e6 thru 0xe36fd85 data_size 0x6(0x3ba0 bytes) 2024-09-18T01:10:09.886 INFO:teuthology.orchestra.run.smithi120.stdout:4300 read 0xa1c19aa thru 0xa1c530c (0x3963 bytes) 2024-09-18T01:10:09.886 INFO:teuthology.orchestra.run.smithi120.stdout:4301 read 0x178b606 thru 0x1793d82 (0x877d bytes) 2024-09-18T01:10:09.886 INFO:teuthology.orchestra.run.smithi120.stdout:4302 writesame 0x6a83dd6 thru 0x6a899aa data_size 0x1(0x5bd5 bytes) 2024-09-18T01:10:09.894 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:09.893+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:09.897 INFO:teuthology.orchestra.run.smithi120.stdout:4303 read 0x7f28ef6 thru 0x7f36136 (0xd241 bytes) 2024-09-18T01:10:09.898 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:09.898 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:09.898 INFO:teuthology.orchestra.run.smithi120.stdout:4306 read 0xe34401f thru 0xe34858d (0x456f bytes) 2024-09-18T01:10:09.898 INFO:teuthology.orchestra.run.smithi120.stdout:4307 punch from 0x43589de to 0x43660bf, (0xd6e1 bytes) 2024-09-18T01:10:09.898 INFO:teuthology.orchestra.run.smithi120.stdout:4308 write 0x8a45dc8 thru 0x8a4fced (0x9f26 bytes) 2024-09-18T01:10:09.906 INFO:teuthology.orchestra.run.smithi120.stdout:4310 read 0x4c30e6b thru 0x4c3eae0 (0xdc76 bytes) 2024-09-18T01:10:09.906 INFO:teuthology.orchestra.run.smithi120.stdout:4311 read 0xd374796 thru 0xd37d4de (0x8d49 bytes) 2024-09-18T01:10:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:4312 punch from 0x2cdcdf7 to 0x2cebe4c, (0xf055 bytes) 2024-09-18T01:10:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:4314 writesame 0x9c267e1 thru 0x9c2ad52 data_size 0x3(0x4572 bytes) 2024-09-18T01:10:09.911 INFO:teuthology.orchestra.run.smithi120.stdout:4315 punch from 0x991b77d to 0x992625a, (0xaadd bytes) 2024-09-18T01:10:09.912 INFO:teuthology.orchestra.run.smithi120.stdout:4317 write 0x6bbd71d thru 0x6bc97fa (0xc0de bytes) 2024-09-18T01:10:09.920 INFO:teuthology.orchestra.run.smithi120.stdout:4318 read 0xe15a2eb thru 0xe1601c1 (0x5ed7 bytes) 2024-09-18T01:10:09.920 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:09.920 INFO:teuthology.orchestra.run.smithi120.stdout:4320 punch from 0x529f6af to 0x52a69e2, (0x7333 bytes) 2024-09-18T01:10:09.920 INFO:teuthology.orchestra.run.smithi120.stdout:4321 writesame 0x56e710f thru 0x56f6776 data_size 0xa6(0xf668 bytes) 2024-09-18T01:10:09.926 INFO:teuthology.orchestra.run.smithi120.stdout:4323 writesame 0xbe8a615 thru 0xbe8adb6 data_size 0x1(0x7a2 bytes) 2024-09-18T01:10:09.929 INFO:teuthology.orchestra.run.smithi120.stdout:4325 write 0x7111653 thru 0x711a844 (0x91f2 bytes) 2024-09-18T01:10:09.936 INFO:teuthology.orchestra.run.smithi120.stdout:4326 write 0x252ab65 thru 0x252ada0 (0x23c bytes) 2024-09-18T01:10:09.939 INFO:teuthology.orchestra.run.smithi120.stdout:4327 punch from 0x97db868 to 0x97e016b, (0x4903 bytes) 2024-09-18T01:10:09.939 INFO:teuthology.orchestra.run.smithi120.stdout:4328 read 0x1e80460 thru 0x1e8193e (0x14df bytes) 2024-09-18T01:10:09.944 INFO:teuthology.orchestra.run.smithi120.stdout:4330 trunc from 0xe36fd86 to 0x9f8d9ae 2024-09-18T01:10:09.954 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:09.954 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:09.959 INFO:teuthology.orchestra.run.smithi120.stdout:4332 writesame 0x935d4bc thru 0x936c3f1 data_size 0x43(0xef36 bytes) 2024-09-18T01:10:09.964 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:09.969 INFO:teuthology.orchestra.run.smithi120.stdout:4335 writesame 0x15d44e thru 0x15e6a0 data_size 0x1(0x1253 bytes) 2024-09-18T01:10:09.973 INFO:teuthology.orchestra.run.smithi120.stdout:4336 write 0x654c2c0 thru 0x65537c7 (0x7508 bytes) 2024-09-18T01:10:09.979 INFO:teuthology.orchestra.run.smithi120.stdout:4337 write 0x8ba7e96 thru 0x8bb7226 (0xf391 bytes) 2024-09-18T01:10:09.985 INFO:teuthology.orchestra.run.smithi120.stdout:4338 write 0x65ac515 thru 0x65b12f1 (0x4ddd bytes) 2024-09-18T01:10:09.989 INFO:teuthology.orchestra.run.smithi120.stdout:4340 write 0x588f8f7 thru 0x58945ea (0x4cf4 bytes) 2024-09-18T01:10:09.995 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:09.995+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:10.004 INFO:teuthology.orchestra.run.smithi120.stdout:4342 writesame 0xdca6e09 thru 0xdcb65ab data_size 0x5(0xf7a3 bytes) 2024-09-18T01:10:10.004 INFO:teuthology.orchestra.run.smithi120.stdout:4343 writesame 0x1c7972a thru 0x1c8203e data_size 0x1(0x8915 bytes) 2024-09-18T01:10:10.020 INFO:teuthology.orchestra.run.smithi120.stdout:4344 write 0x2105085 thru 0x2105339 (0x2b5 bytes) 2024-09-18T01:10:10.023 INFO:teuthology.orchestra.run.smithi120.stdout:4345 read 0x49fe2e2 thru 0x4a024fd (0x421c bytes) 2024-09-18T01:10:10.023 INFO:teuthology.orchestra.run.smithi120.stdout:4346 write 0x84b986c thru 0x84c3c0c (0xa3a1 bytes) 2024-09-18T01:10:10.028 INFO:teuthology.orchestra.run.smithi120.stdout:4347 write 0x956b8e7 thru 0x956dcbf (0x23d9 bytes) 2024-09-18T01:10:10.032 INFO:teuthology.orchestra.run.smithi120.stdout:4348 writesame 0x90449d8 thru 0x90547a9 data_size 0x35(0xfdd2 bytes) 2024-09-18T01:10:10.039 INFO:teuthology.orchestra.run.smithi120.stdout:4349 read 0xb6ae7a6 thru 0xb6af010 (0x86b bytes) 2024-09-18T01:10:10.043 INFO:teuthology.orchestra.run.smithi120.stdout:4350 trunc from 0xdcb65ac to 0x9ed7106 2024-09-18T01:10:10.050 INFO:teuthology.orchestra.run.smithi120.stdout:4351 read 0x4f3c391 thru 0x4f44a8a (0x86fa bytes) 2024-09-18T01:10:10.051 INFO:teuthology.orchestra.run.smithi120.stdout:4352 read 0x38fc8ba thru 0x390a1e9 (0xd930 bytes) 2024-09-18T01:10:10.051 INFO:teuthology.orchestra.run.smithi120.stdout:4353 trunc from 0x9ed7106 to 0x9139139 2024-09-18T01:10:10.072 INFO:teuthology.orchestra.run.smithi120.stdout:4354 write 0x225ebdd thru 0x2267bcc (0x8ff0 bytes) 2024-09-18T01:10:10.077 INFO:teuthology.orchestra.run.smithi120.stdout:4356 read 0x3f901e thru 0x3fb4bb (0x249e bytes) 2024-09-18T01:10:10.078 INFO:teuthology.orchestra.run.smithi120.stdout:4357 punch from 0x2f14443 to 0x2f2306d, (0xec2a bytes) 2024-09-18T01:10:10.078 INFO:teuthology.orchestra.run.smithi120.stdout:4358 read 0x72ac3f5 thru 0x72b4150 (0x7d5c bytes) 2024-09-18T01:10:10.079 INFO:teuthology.orchestra.run.smithi120.stdout:4359 trunc from 0x9139139 to 0xa1385f0 2024-09-18T01:10:10.085 INFO:teuthology.orchestra.run.smithi120.stdout:4360 writesame 0x37a9fe7 thru 0x37b37f1 data_size 0x1(0x980b bytes) 2024-09-18T01:10:10.096 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:10.095+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:10.102 INFO:teuthology.orchestra.run.smithi120.stdout:4361 punch from 0x6c24ca1 to 0x6c27aaf, (0x2e0e bytes) 2024-09-18T01:10:10.102 INFO:teuthology.orchestra.run.smithi120.stdout:4362 read 0x94fcf71 thru 0x9500f21 (0x3fb1 bytes) 2024-09-18T01:10:10.103 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.103 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.112 INFO:teuthology.orchestra.run.smithi120.stdout:4367 writesame 0xe5541e6 thru 0xe557f74 data_size 0x33(0x3d8f bytes) 2024-09-18T01:10:10.113 INFO:teuthology.orchestra.run.smithi120.stdout:4369 punch from 0x543469a to 0x5434ce6, (0x64c bytes) 2024-09-18T01:10:10.113 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.120 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:10.120 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:10.122 INFO:teuthology.orchestra.run.smithi120.stdout:4372 trunc from 0xe557f75 to 0x5cbdf1b 2024-09-18T01:10:10.131 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:10.151 INFO:teuthology.orchestra.run.smithi120.stdout:4374 write 0x36196c2 thru 0x3619cdd (0x61c bytes) 2024-09-18T01:10:10.153 INFO:teuthology.orchestra.run.smithi120.stdout:4375 punch from 0x1f0321f to 0x1f03b20, (0x901 bytes) 2024-09-18T01:10:10.154 INFO:teuthology.orchestra.run.smithi120.stdout:4376 write 0x3c4e679 thru 0x3c50f8f (0x2917 bytes) 2024-09-18T01:10:10.161 INFO:teuthology.orchestra.run.smithi120.stdout:4377 write 0x4430fbe thru 0x4433682 (0x26c5 bytes) 2024-09-18T01:10:10.164 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.164 INFO:teuthology.orchestra.run.smithi120.stdout:4379 write 0xbca630 thru 0xbcf4e8 (0x4eb9 bytes) 2024-09-18T01:10:10.176 INFO:teuthology.orchestra.run.smithi120.stdout:4380 write 0x93ddaec thru 0x93dde02 (0x317 bytes) 2024-09-18T01:10:10.180 INFO:teuthology.orchestra.run.smithi120.stdout:4381 read 0xed2d5e thru 0xedd00b (0xa2ae bytes) 2024-09-18T01:10:10.180 INFO:teuthology.orchestra.run.smithi120.stdout:4382 read 0x82b9c76 thru 0x82be1fc (0x4587 bytes) 2024-09-18T01:10:10.185 INFO:teuthology.orchestra.run.smithi120.stdout:4383 trunc from 0x93dde03 to 0x4540398 2024-09-18T01:10:10.197 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:10.196+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:10.220 INFO:teuthology.orchestra.run.smithi120.stdout:4384 writesame 0xd0f3c7d thru 0xd0fecb0 data_size 0x7e(0xb034 bytes) 2024-09-18T01:10:10.220 INFO:teuthology.orchestra.run.smithi120.stdout:4385 read 0x76000a2 thru 0x7609632 (0x9591 bytes) 2024-09-18T01:10:10.221 INFO:teuthology.orchestra.run.smithi120.stdout:4386 read 0x8c455a2 thru 0x8c4f43e (0x9e9d bytes) 2024-09-18T01:10:10.221 INFO:teuthology.orchestra.run.smithi120.stdout:4387 writesame 0x2035f7c thru 0x2038645 data_size 0x1(0x26ca bytes) 2024-09-18T01:10:10.227 INFO:teuthology.orchestra.run.smithi120.stdout:4388 writesame 0x7e7f375 thru 0x7e84256 data_size 0x2e(0x4ee2 bytes) 2024-09-18T01:10:10.236 INFO:teuthology.orchestra.run.smithi120.stdout:4390 write 0x3ec5ed4 thru 0x3ed592d (0xfa5a bytes) 2024-09-18T01:10:10.243 INFO:teuthology.orchestra.run.smithi120.stdout:4391 read 0x40586d6 thru 0x4065785 (0xd0b0 bytes) 2024-09-18T01:10:10.244 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:10.244 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:10.248 INFO:teuthology.orchestra.run.smithi120.stdout:4392 write 0xd30df91 thru 0xd30f744 (0x17b4 bytes) 2024-09-18T01:10:10.251 INFO:teuthology.orchestra.run.smithi120.stdout:4393 read 0xa7eca06 thru 0xa7fc944 (0xff3f bytes) 2024-09-18T01:10:10.251 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.251 INFO:teuthology.orchestra.run.smithi120.stdout:4396 read 0x71c3e16 thru 0x71d3325 (0xf510 bytes) 2024-09-18T01:10:10.251 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:10.265 INFO:teuthology.orchestra.run.smithi120.stdout:4398 trunc from 0xd30f745 to 0x136fdd6 2024-09-18T01:10:10.291 INFO:teuthology.orchestra.run.smithi120.stdout:4399 read 0xe2283b thru 0xe2cf65 (0xa72b bytes) 2024-09-18T01:10:10.291 INFO:teuthology.orchestra.run.smithi120.stdout:4402 read 0x9d6dea thru 0x9d8395 (0x15ac bytes) 2024-09-18T01:10:10.291 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.312 INFO:teuthology.orchestra.run.smithi120.stdout:4406 write 0xe54fe6c thru 0xe554b9b (0x4d30 bytes) 2024-09-18T01:10:10.318 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.318 INFO:teuthology.orchestra.run.smithi120.stdout:4408 punch from 0x95ebbd0 to 0x95f628d, (0xa6bd bytes) 2024-09-18T01:10:10.318 INFO:teuthology.orchestra.run.smithi120.stdout:4409 writesame 0x26138e thru 0x26f1e2 data_size 0x2f(0xde55 bytes) 2024-09-18T01:10:10.322 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.322 INFO:teuthology.orchestra.run.smithi120.stdout:4413 write 0x48a352d thru 0x48aefdd (0xbab1 bytes) 2024-09-18T01:10:10.330 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.330 INFO:teuthology.orchestra.run.smithi120.stdout:4415 write 0xbc2280f thru 0xbc26c0e (0x4400 bytes) 2024-09-18T01:10:10.333 INFO:teuthology.orchestra.run.smithi120.stdout:4417 read 0x1d0bb16 thru 0x1d0cbda (0x10c5 bytes) 2024-09-18T01:10:10.334 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.334 INFO:teuthology.orchestra.run.smithi120.stdout:4419 writesame 0x1314d3d thru 0x1319a3d data_size 0x1(0x4d01 bytes) 2024-09-18T01:10:10.338 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:10.337+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:10.343 INFO:teuthology.orchestra.run.smithi120.stdout:4420 write 0x94680c1 thru 0x9470985 (0x88c5 bytes) 2024-09-18T01:10:10.352 INFO:teuthology.orchestra.run.smithi120.stdout:4421 punch from 0xd9671b4 to 0xd96ee2c, (0x7c78 bytes) 2024-09-18T01:10:10.364 INFO:teuthology.orchestra.run.smithi120.stdout:4422 trunc from 0xe554b9c to 0x2a6cd11 2024-09-18T01:10:10.383 INFO:teuthology.orchestra.run.smithi120.stdout:4423 punch from 0x99b915 to 0x9a47d0, (0x8ebb bytes) 2024-09-18T01:10:10.384 INFO:teuthology.orchestra.run.smithi120.stdout:4426 punch from 0x4f74f6 to 0x502427, (0xaf31 bytes) 2024-09-18T01:10:10.394 INFO:teuthology.orchestra.run.smithi120.stdout:4427 write 0x6a441ed thru 0x6a47580 (0x3394 bytes) 2024-09-18T01:10:10.399 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:10.398+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:10.407 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:10.407 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:10.409 INFO:teuthology.orchestra.run.smithi120.stdout:4428 write 0xa2e9b48 thru 0xa2f5f17 (0xc3d0 bytes) 2024-09-18T01:10:10.416 INFO:teuthology.orchestra.run.smithi120.stdout:4430 read 0x120cffa thru 0x121362e (0x6635 bytes) 2024-09-18T01:10:10.418 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:10.421 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:10.422 INFO:teuthology.orchestra.run.smithi120.stdout:4431 write 0xac68017 thru 0xac6e34b (0x6335 bytes) 2024-09-18T01:10:10.427 INFO:teuthology.orchestra.run.smithi120.stdout:4432 read 0x8934574 thru 0x89361ee (0x1c7b bytes) 2024-09-18T01:10:10.427 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:10.431 INFO:teuthology.orchestra.run.smithi120.stdout:4436 trunc from 0xac6e34c to 0x8074ff4 2024-09-18T01:10:10.445 INFO:teuthology.orchestra.run.smithi120.stdout:4437 write 0x7a2abb3 thru 0x7a2ce9c (0x22ea bytes) 2024-09-18T01:10:10.449 INFO:teuthology.orchestra.run.smithi120.stdout:4438 clone 38 order 20 su 524288 sc 10 2024-09-18T01:10:10.502 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:10.501+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:10.542 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:10.543 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:10.559 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:10.559 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:10.562 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:10.562 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:10.571 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:10.575 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:10.591 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:10.591 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:10.600 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:10.599+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:10.608 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:10.608 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:10.625 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:10.625 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:10.646 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:10.646 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:10.665 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:10.665 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:10.718 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:10.717+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:10.722 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:10.722 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:10.722 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:10.722 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:10.722 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:10.730 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:10.802 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:10.801+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 178.937232255, 2024-09-18T01:10:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.213303885000002, 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.213281703999996, 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 41.245777236999999, 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.240804907999999, 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.235912525, 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.230497887, 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.218923621, 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:10.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.209955927999999, 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.884 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:10.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.209901466, 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:10.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:10.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.204630092, 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:10.898 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.899 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:10.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:10.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:10.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.904 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.204549353000001, 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.905 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:10.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.910 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.203588354000001, 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:10.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.203529562, 2024-09-18T01:10:10.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:10.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:10.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:10.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.924 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.202365594, 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:10.925 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:10.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.202000925, 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:10.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:10.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.937 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.201478389, 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.201081007999999, 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:10.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.199228278, 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:10.948 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:10.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:10.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:10.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.199106087000001, 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:10.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:10.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:10.959 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:10.960 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:10.959+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:10.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:11.062 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:11.061+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:11.085 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:11.085 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:11.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:11.162 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:11.162+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:11.189 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:11.189 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:11.197 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:11.282 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:11.281+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 133.619297147, 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.595839056999999, 2024-09-18T01:10:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582760804999999, 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582740548, 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582395513, 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582088383, 2024-09-18T01:10:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582052623999999, 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.582014355, 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581989883, 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581969222, 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581950598000001, 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581930806999999, 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581911703999999, 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581893614, 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581875515, 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581857186000001, 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.58183882, 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581820560000001, 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581802106, 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.581752420000001, 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.457 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:11.459 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:11.459 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:11.457+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:11.560 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:11.559+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:11.581 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:10:11.603 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:11.603 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:11.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:11.697 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:11.696+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:11.761 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:11.760+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:11.787 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:11.787 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:11.796 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:11.862 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:11.861+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:11.941 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:11.941 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.027686299999999, 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:11.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:11.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.027478269, 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:11.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:11.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013441694000001, 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:11.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013418346, 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:11.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013396824999999, 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.965 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013376106000001, 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:11.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.01335533, 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013335185000001, 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013314809000001, 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013294081, 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013269622999999, 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:11.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.013249146, 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:11.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:11.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:11.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.00447295, 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:11.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:12.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:12.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004447949999999, 2024-09-18T01:10:12.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:12.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:12.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004410795, 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:12.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004384112, 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:12.012 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:12.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004364213000001, 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:12.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:12.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004345005999999, 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:12.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.004325993, 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:12.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.00430772, 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:12.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:12.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.033 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.035 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:12.035 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:12.034+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:12.136 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:12.135+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:12.178 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:12.179 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:12.179 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:12.179 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:12.237 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:12.236+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:12.291 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:12.292 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.300 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:12.344 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:12.343+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:12.403 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:12.403 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:12.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:12.403 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.740687541, 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:12.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.740633316, 2024-09-18T01:10:12.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:12.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:12.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.740600858000001, 2024-09-18T01:10:12.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:12.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.740469595, 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.420 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:12.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.424 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.739999609000002, 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:12.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.738837009000001, 2024-09-18T01:10:12.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:12.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.434 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.738729726999999, 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.435 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:12.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:12.440 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.738648068000003, 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:12.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:12.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.73813268, 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:12.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.737313442000001, 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.164048356000002, 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.997189464000002, 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:12.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:12.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.969344075999999, 2024-09-18T01:10:12.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.468 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.969323777, 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:12.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.480 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.969303514, 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:12.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.969283660999999, 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:12.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:12.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.969264504000002, 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.493 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:12.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.585274078000001, 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:12.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:12.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.525315214999999, 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:12.505 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.506 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:12.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:12.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.74081176900000001, 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:12.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:12.517 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.518 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:12.519 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:12.517+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:12.619 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:12.618+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:12.633 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:10:12.669 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:12.669 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:12.669 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:12.669 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.677 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:12.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:12.720 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:12.719+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:12.738 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-18T01:10:12.815 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:12.821 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:12.819+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:12.832 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:12.840 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:12.840 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:12.840 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:12.840 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:12.840 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:12.850 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:12.850 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:12.851 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:12.866 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:12.884 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:12.884 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:12.901 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:12.901 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:12.918 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:12.919 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:12.921 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:12.920+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:12.936 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:12.958 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:12.958 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:12.967 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:13.000 INFO:teuthology.orchestra.run.smithi120.stdout:4441 trunc from 0x8074ff4 to 0x1a88228 2024-09-18T01:10:13.020 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.021 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:13.020+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:13.028 INFO:teuthology.orchestra.run.smithi120.stdout:4443 trunc from 0x1a88228 to 0x8941cb9 2024-09-18T01:10:13.034 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.035 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.035 INFO:teuthology.orchestra.run.smithi120.stdout:4446 read 0x86b6478 thru 0x86bf7de (0x9367 bytes) 2024-09-18T01:10:13.035 INFO:teuthology.orchestra.run.smithi120.stdout:4448 writesame 0x3e504fb thru 0x3e56740 data_size 0x15(0x6246 bytes) 2024-09-18T01:10:13.048 INFO:teuthology.orchestra.run.smithi120.stdout:4449 write 0xb784cc9 thru 0xb78a8f3 (0x5c2b bytes) 2024-09-18T01:10:13.052 INFO:teuthology.orchestra.run.smithi120.stdout:4450 writesame 0xa5d96a0 thru 0xa5e43ec data_size 0x13(0xad4d bytes) 2024-09-18T01:10:13.059 INFO:teuthology.orchestra.run.smithi120.stdout:4451 trunc from 0xb78a8f4 to 0x9c1bd77 2024-09-18T01:10:13.073 INFO:teuthology.orchestra.run.smithi120.stdout:4452 punch from 0x2fb8338 to 0x2fc2f02, (0xabca bytes) 2024-09-18T01:10:13.073 INFO:teuthology.orchestra.run.smithi120.stdout:4454 writesame 0x99ae9aa thru 0x99b04d1 data_size 0x4(0x1b28 bytes) 2024-09-18T01:10:13.077 INFO:teuthology.orchestra.run.smithi120.stdout:4455 punch from 0x3924784 to 0x3931e11, (0xd68d bytes) 2024-09-18T01:10:13.078 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.078 INFO:teuthology.orchestra.run.smithi120.stdout:4457 writesame 0x82be9d8 thru 0x82c8e4f data_size 0x4c(0xa478 bytes) 2024-09-18T01:10:13.088 INFO:teuthology.orchestra.run.smithi120.stdout:4458 trunc from 0x9c1bd77 to 0xe16b9f9 2024-09-18T01:10:13.090 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.090 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.094 INFO:teuthology.orchestra.run.smithi120.stdout:4459 read 0xd15cfeb thru 0xd15f1e5 (0x21fb bytes) 2024-09-18T01:10:13.094 INFO:teuthology.orchestra.run.smithi120.stdout:4460 read 0x3c170c5 thru 0x3c1f228 (0x8164 bytes) 2024-09-18T01:10:13.094 INFO:teuthology.orchestra.run.smithi120.stdout:4461 read 0x871556d thru 0x8717d44 (0x27d8 bytes) 2024-09-18T01:10:13.094 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.094 INFO:teuthology.orchestra.run.smithi120.stdout:4463 writesame 0xba83d22 thru 0xba8fa11 data_size 0x8(0xbcf0 bytes) 2024-09-18T01:10:13.099 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:13.100 INFO:teuthology.orchestra.run.smithi120.stdout:4465 write 0x1da8501 thru 0x1dac7e7 (0x42e7 bytes) 2024-09-18T01:10:13.104 INFO:teuthology.orchestra.run.smithi120.stdout:4466 read 0xd435f0a thru 0xd43ff6a (0xa061 bytes) 2024-09-18T01:10:13.104 INFO:teuthology.orchestra.run.smithi120.stdout:4467 write 0x460e4cc thru 0x4616864 (0x8399 bytes) 2024-09-18T01:10:13.107 INFO:teuthology.orchestra.run.smithi120.stdout:4469 write 0xa9f7e40 thru 0xa9ff0b0 (0x7271 bytes) 2024-09-18T01:10:13.111 INFO:teuthology.orchestra.run.smithi120.stdout:4470 read 0xa1f5aed thru 0xa1feeeb (0x93ff bytes) 2024-09-18T01:10:13.112 INFO:teuthology.orchestra.run.smithi120.stdout:4471 punch from 0xa3b32c0 to 0xa3c1641, (0xe381 bytes) 2024-09-18T01:10:13.112 INFO:teuthology.orchestra.run.smithi120.stdout:4472 read 0x45cff5e thru 0x45de2b6 (0xe359 bytes) 2024-09-18T01:10:13.112 INFO:teuthology.orchestra.run.smithi120.stdout:4473 writesame 0xb3b614f thru 0xb3c0c07 data_size 0x1(0xaab9 bytes) 2024-09-18T01:10:13.132 INFO:teuthology.orchestra.run.smithi120.stdout:4474 write 0xd38d410 thru 0xd3977b1 (0xa3a2 bytes) 2024-09-18T01:10:13.136 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.137 INFO:teuthology.orchestra.run.smithi120.stdout:4476 trunc from 0xe16b9f9 to 0xc86dc19 2024-09-18T01:10:13.140 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:13.139+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:13.154 INFO:teuthology.orchestra.run.smithi120.stdout:4477 writesame 0xd2437c9 thru 0xd243f31 data_size 0x1(0x769 bytes) 2024-09-18T01:10:13.154 INFO:teuthology.orchestra.run.smithi120.stdout:4479 read 0xcd439a9 thru 0xcd48762 (0x4dba bytes) 2024-09-18T01:10:13.154 INFO:teuthology.orchestra.run.smithi120.stdout:4480 read 0x67fa7b8 thru 0x680a66d (0xfeb6 bytes) 2024-09-18T01:10:13.154 INFO:teuthology.orchestra.run.smithi120.stdout:4481 read 0x108a5b4 thru 0x108debe (0x390b bytes) 2024-09-18T01:10:13.154 INFO:teuthology.orchestra.run.smithi120.stdout:4484 read 0x39aa7c thru 0x3a2a32 (0x7fb7 bytes) 2024-09-18T01:10:13.155 INFO:teuthology.orchestra.run.smithi120.stdout:4485 write 0x78d11dd thru 0x78d7a53 (0x6877 bytes) 2024-09-18T01:10:13.158 INFO:teuthology.orchestra.run.smithi120.stdout:4486 writesame 0x9803f9a thru 0x9807a95 data_size 0x32(0x3afc bytes) 2024-09-18T01:10:13.162 INFO:teuthology.orchestra.run.smithi120.stdout:4487 read 0xa9c8e49 thru 0xa9d5cca (0xce82 bytes) 2024-09-18T01:10:13.162 INFO:teuthology.orchestra.run.smithi120.stdout:4488 write 0x184a3ba thru 0x18540f5 (0x9d3c bytes) 2024-09-18T01:10:13.166 INFO:teuthology.orchestra.run.smithi120.stdout:4489 punch from 0xd3ebed to 0xd49c13, (0xb026 bytes) 2024-09-18T01:10:13.170 INFO:teuthology.orchestra.run.smithi120.stdout:4490 trunc from 0xd243f32 to 0x9af56b2 2024-09-18T01:10:13.188 INFO:teuthology.orchestra.run.smithi120.stdout:4491 trunc from 0x9af56b2 to 0x5484911 2024-09-18T01:10:13.213 INFO:teuthology.orchestra.run.smithi120.stdout:4494 write 0xa75c9a4 thru 0xa7653f6 (0x8a53 bytes) 2024-09-18T01:10:13.216 INFO:teuthology.orchestra.run.smithi120.stdout:4495 read 0x10d158e thru 0x10d7ea1 (0x6914 bytes) 2024-09-18T01:10:13.218 INFO:teuthology.orchestra.run.smithi120.stdout:4497 trunc from 0xa7653f7 to 0xb97970c 2024-09-18T01:10:13.223 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:13.222+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:13.224 INFO:teuthology.orchestra.run.smithi120.stdout:4498 write 0x129b5b4 thru 0x12ab55a (0xffa7 bytes) 2024-09-18T01:10:13.227 INFO:teuthology.orchestra.run.smithi120.stdout:4499 punch from 0x6ceda91 to 0x6cede5f, (0x3ce bytes) 2024-09-18T01:10:13.227 INFO:teuthology.orchestra.run.smithi120.stdout:4501 read 0xef53cb thru 0xf0276d (0xd3a3 bytes) 2024-09-18T01:10:13.227 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.235 INFO:teuthology.orchestra.run.smithi120.stdout:4503 write 0xd2fc1cd thru 0xd2fd1ad (0xfe1 bytes) 2024-09-18T01:10:13.238 INFO:teuthology.orchestra.run.smithi120.stdout:4504 write 0x7976c0b thru 0x7982f6c (0xc362 bytes) 2024-09-18T01:10:13.242 INFO:teuthology.orchestra.run.smithi120.stdout:4505 punch from 0x8b6c880 to 0x8b79332, (0xcab2 bytes) 2024-09-18T01:10:13.242 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.243 INFO:teuthology.orchestra.run.smithi120.stdout:4509 trunc from 0xd2fd1ae to 0xdc8137b 2024-09-18T01:10:13.249 INFO:teuthology.orchestra.run.smithi120.stdout:4510 read 0x6ab09ef thru 0x6abfd50 (0xf362 bytes) 2024-09-18T01:10:13.252 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.252 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:13.264 INFO:teuthology.orchestra.run.smithi120.stdout:4512 trunc from 0xdc8137b to 0x64d0a6 2024-09-18T01:10:13.282 INFO:teuthology.orchestra.run.smithi120.stdout:4513 read 0x23848a thru 0x242e77 (0xa9ee bytes) 2024-09-18T01:10:13.290 INFO:teuthology.orchestra.run.smithi120.stdout:4514 write 0x14e7101 thru 0x14f1aeb (0xa9eb bytes) 2024-09-18T01:10:13.294 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.294 INFO:teuthology.orchestra.run.smithi120.stdout:4516 punch from 0x46fb68 to 0x476cec, (0x7184 bytes) 2024-09-18T01:10:13.296 INFO:teuthology.orchestra.run.smithi120.stdout:4517 trunc from 0x14f1aec to 0x3a8a767 2024-09-18T01:10:13.302 INFO:teuthology.orchestra.run.smithi120.stdout:4518 punch from 0x757086 to 0x75e38a, (0x7304 bytes) 2024-09-18T01:10:13.321 INFO:teuthology.orchestra.run.smithi120.stdout:4519 write 0xc2b02e6 thru 0xc2bda68 (0xd783 bytes) 2024-09-18T01:10:13.327 INFO:teuthology.orchestra.run.smithi120.stdout:4520 read 0x62d9c3 thru 0x638a0d (0xb04b bytes) 2024-09-18T01:10:13.328 INFO:teuthology.orchestra.run.smithi120.stdout:4521 read 0xb8da403 thru 0xb8e7b4f (0xd74d bytes) 2024-09-18T01:10:13.328 INFO:teuthology.orchestra.run.smithi120.stdout:4523 write 0x77194d4 thru 0x7723fab (0xaad8 bytes) 2024-09-18T01:10:13.334 INFO:teuthology.orchestra.run.smithi120.stdout:4524 trunc from 0xc2bda69 to 0xb0eb036 2024-09-18T01:10:13.346 INFO:teuthology.orchestra.run.smithi120.stdout:4525 punch from 0x714e87e to 0x715c42e, (0xdbb0 bytes) 2024-09-18T01:10:13.346 INFO:teuthology.orchestra.run.smithi120.stdout:4526 punch from 0x3571e3f to 0x3576c63, (0x4e24 bytes) 2024-09-18T01:10:13.346 INFO:teuthology.orchestra.run.smithi120.stdout:4527 read 0x2045e5b thru 0x204c894 (0x6a3a bytes) 2024-09-18T01:10:13.347 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:13.346+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:13.348 INFO:teuthology.orchestra.run.smithi120.stdout:4528 trunc from 0xb0eb036 to 0x926b142 2024-09-18T01:10:13.361 INFO:teuthology.orchestra.run.smithi120.stdout:4531 write 0xa75d84e thru 0xa75d928 (0xdb bytes) 2024-09-18T01:10:13.366 INFO:teuthology.orchestra.run.smithi120.stdout:4532 trunc from 0xa75d929 to 0xbb6e8d8 2024-09-18T01:10:13.379 INFO:teuthology.orchestra.run.smithi120.stdout:4539 write 0xca2a48a thru 0xca2f821 (0x5398 bytes) 2024-09-18T01:10:13.387 INFO:teuthology.orchestra.run.smithi120.stdout:4541 trunc from 0xca2f822 to 0x94aa895 2024-09-18T01:10:13.409 INFO:teuthology.orchestra.run.smithi120.stdout:4542 write 0xa3374e3 thru 0xa33e7d1 (0x72ef bytes) 2024-09-18T01:10:13.413 INFO:teuthology.orchestra.run.smithi120.stdout:4543 write 0x4813088 thru 0x48143a7 (0x1320 bytes) 2024-09-18T01:10:13.414 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.414 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.416 INFO:teuthology.orchestra.run.smithi120.stdout:4545 punch from 0x2609dc0 to 0x260adac, (0xfec bytes) 2024-09-18T01:10:13.416 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.423 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:13.424 INFO:teuthology.orchestra.run.smithi120.stdout:4547 trunc from 0xa33e7d2 to 0x448bd94 2024-09-18T01:10:13.439 INFO:teuthology.orchestra.run.smithi120.stdout:4548 punch from 0x12161da to 0x121805b, (0x1e81 bytes) 2024-09-18T01:10:13.448 INFO:teuthology.orchestra.run.smithi120.stdout:4549 trunc from 0x448bd94 to 0xc115f4a 2024-09-18T01:10:13.457 INFO:teuthology.orchestra.run.smithi120.stdout:4550 punch from 0x6960a16 to 0x696e707, (0xdcf1 bytes) 2024-09-18T01:10:13.457 INFO:teuthology.orchestra.run.smithi120.stdout:4551 read 0x5d639a5 thru 0x5d6c639 (0x8c95 bytes) 2024-09-18T01:10:13.457 INFO:teuthology.orchestra.run.smithi120.stdout:4552 writesame 0x4dadd06 thru 0x4db065d data_size 0x31(0x2958 bytes) 2024-09-18T01:10:13.460 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.460 INFO:teuthology.orchestra.run.smithi120.stdout:4556 punch from 0x52bd1e3 to 0x52cb5e0, (0xe3fd bytes) 2024-09-18T01:10:13.464 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:13.463+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:13.470 INFO:teuthology.orchestra.run.smithi120.stdout:4558 trunc from 0xc115f4a to 0x34ad9ed 2024-09-18T01:10:13.485 INFO:teuthology.orchestra.run.smithi120.stdout:4559 trunc from 0x34ad9ed to 0x6306d30 2024-09-18T01:10:13.492 INFO:teuthology.orchestra.run.smithi120.stdout:4561 write 0x5fc0ae4 thru 0x5fc9e57 (0x9374 bytes) 2024-09-18T01:10:13.496 INFO:teuthology.orchestra.run.smithi120.stdout:4562 read 0x52f97fb thru 0x530694f (0xd155 bytes) 2024-09-18T01:10:13.510 INFO:teuthology.orchestra.run.smithi120.stdout:4563 write 0xc36ea6c thru 0xc379600 (0xab95 bytes) 2024-09-18T01:10:13.513 INFO:teuthology.orchestra.run.smithi120.stdout:4564 writesame 0x18086e9 thru 0x18096f9 data_size 0x3(0x1011 bytes) 2024-09-18T01:10:13.517 INFO:teuthology.orchestra.run.smithi120.stdout:4565 write 0x31755d5 thru 0x3183a38 (0xe464 bytes) 2024-09-18T01:10:13.523 INFO:teuthology.orchestra.run.smithi120.stdout:4566 punch from 0x3fe18a3 to 0x3fecd9d, (0xb4fa bytes) 2024-09-18T01:10:13.525 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:13.525+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:13.526 INFO:teuthology.orchestra.run.smithi120.stdout:4567 trunc from 0xc379601 to 0xec2556f 2024-09-18T01:10:13.535 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.535 INFO:teuthology.orchestra.run.smithi120.stdout:4571 read 0x2340e31 thru 0x234787b (0x6a4b bytes) 2024-09-18T01:10:13.535 INFO:teuthology.orchestra.run.smithi120.stdout:4572 write 0x903e44c thru 0x904783c (0x93f1 bytes) 2024-09-18T01:10:13.538 INFO:teuthology.orchestra.run.smithi120.stdout:4575 read 0x29afc6 thru 0x2a89ee (0xda29 bytes) 2024-09-18T01:10:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:4576 read 0x92085fb thru 0x921140b (0x8e11 bytes) 2024-09-18T01:10:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:4578 punch from 0x787701b to 0x787da31, (0x6a16 bytes) 2024-09-18T01:10:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:4581 write 0x48ade85 thru 0x48b8157 (0xa2d3 bytes) 2024-09-18T01:10:13.542 INFO:teuthology.orchestra.run.smithi120.stdout:4582 write 0x23d3602 thru 0x23d5a75 (0x2474 bytes) 2024-09-18T01:10:13.546 INFO:teuthology.orchestra.run.smithi120.stdout:4583 read 0x9792c72 thru 0x9796101 (0x3490 bytes) 2024-09-18T01:10:13.546 INFO:teuthology.orchestra.run.smithi120.stdout:4584 read 0x1e9cc70 thru 0x1ea366d (0x69fe bytes) 2024-09-18T01:10:13.561 INFO:teuthology.orchestra.run.smithi120.stdout:4585 trunc from 0xec2556f to 0x1f2f290 2024-09-18T01:10:13.574 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.574 INFO:teuthology.orchestra.run.smithi120.stdout:4587 read 0xde1b93 thru 0xde3727 (0x1b95 bytes) 2024-09-18T01:10:13.574 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.575 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.575 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.584 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:13.593 INFO:teuthology.orchestra.run.smithi120.stdout:4592 writesame 0xd3101ca thru 0xd319a13 data_size 0x65(0x984a bytes) 2024-09-18T01:10:13.593 INFO:teuthology.orchestra.run.smithi120.stdout:4593 writesame 0x97dd496 thru 0x97eb0b0 data_size 0x1d(0xdc1b bytes) 2024-09-18T01:10:13.597 INFO:teuthology.orchestra.run.smithi120.stdout:4594 writesame 0xa5c7a73 thru 0xa5d1594 data_size 0x3(0x9b22 bytes) 2024-09-18T01:10:13.607 INFO:teuthology.orchestra.run.smithi120.stdout:4596 read 0xd0536d7 thru 0xd055941 (0x226b bytes) 2024-09-18T01:10:13.607 INFO:teuthology.orchestra.run.smithi120.stdout:4597 read 0x49364e7 thru 0x49424f8 (0xc012 bytes) 2024-09-18T01:10:13.614 INFO:teuthology.orchestra.run.smithi120.stdout:4598 trunc from 0xd319a14 to 0x7319a22 2024-09-18T01:10:13.625 INFO:teuthology.orchestra.run.smithi120.stdout:4599 read 0xd64410 thru 0xd70225 (0xbe16 bytes) 2024-09-18T01:10:13.625 INFO:teuthology.orchestra.run.smithi120.stdout:4600 read 0x1c55a11 thru 0x1c64970 (0xef60 bytes) 2024-09-18T01:10:13.628 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:13.627+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:13.635 INFO:teuthology.orchestra.run.smithi120.stdout:4601 write 0xb0d7537 thru 0xb0e3a50 (0xc51a bytes) 2024-09-18T01:10:13.639 INFO:teuthology.orchestra.run.smithi120.stdout:4602 punch from 0x641265e to 0x641d59f, (0xaf41 bytes) 2024-09-18T01:10:13.639 INFO:teuthology.orchestra.run.smithi120.stdout:4603 write 0x6fc32bc thru 0x6fce774 (0xb4b9 bytes) 2024-09-18T01:10:13.645 INFO:teuthology.orchestra.run.smithi120.stdout:4605 punch from 0x69101f8 to 0x6919a32, (0x983a bytes) 2024-09-18T01:10:13.645 INFO:teuthology.orchestra.run.smithi120.stdout:4606 write 0x3a636d6 thru 0x3a6f5bd (0xbee8 bytes) 2024-09-18T01:10:13.649 INFO:teuthology.orchestra.run.smithi120.stdout:4607 read 0x1ee2354 thru 0x1ee8a4f (0x66fc bytes) 2024-09-18T01:10:13.650 INFO:teuthology.orchestra.run.smithi120.stdout:4610 read 0x159d9ca thru 0x15a3cc8 (0x62ff bytes) 2024-09-18T01:10:13.650 INFO:teuthology.orchestra.run.smithi120.stdout:4611 read 0x3ce1bb6 thru 0x3ceae3e (0x9289 bytes) 2024-09-18T01:10:13.650 INFO:teuthology.orchestra.run.smithi120.stdout:4612 writesame 0x2edb0c8 thru 0x2ede0ca data_size 0x11(0x3003 bytes) 2024-09-18T01:10:13.659 INFO:teuthology.orchestra.run.smithi120.stdout:4613 trunc from 0xb0e3a51 to 0x5da7b7c 2024-09-18T01:10:13.670 INFO:teuthology.orchestra.run.smithi120.stdout:4614 read 0x4a0543a thru 0x4a0b72b (0x62f2 bytes) 2024-09-18T01:10:13.670 INFO:teuthology.orchestra.run.smithi120.stdout:4615 punch from 0x1440b7a to 0x144ea7f, (0xdf05 bytes) 2024-09-18T01:10:13.670 INFO:teuthology.orchestra.run.smithi120.stdout:4616 punch from 0x592269 to 0x59d6ca, (0xb461 bytes) 2024-09-18T01:10:13.676 INFO:teuthology.orchestra.run.smithi120.stdout:4617 trunc from 0x5da7b7c to 0xb33384c 2024-09-18T01:10:13.683 INFO:teuthology.orchestra.run.smithi120.stdout:4618 write 0x14360bd thru 0x143d6d2 (0x7616 bytes) 2024-09-18T01:10:13.687 INFO:teuthology.orchestra.run.smithi120.stdout:4620 read 0x48ca6c6 thru 0x48d7176 (0xcab1 bytes) 2024-09-18T01:10:13.687 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.687 INFO:teuthology.orchestra.run.smithi120.stdout:4622 write 0x4a9837a thru 0x4a9fbaa (0x7831 bytes) 2024-09-18T01:10:13.694 INFO:teuthology.orchestra.run.smithi120.stdout:4624 trunc from 0xb33384c to 0x7fcf6b4 2024-09-18T01:10:13.698 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.699 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:13.711 INFO:teuthology.orchestra.run.smithi120.stdout:4625 write 0xc89a53f thru 0xc89d9a5 (0x3467 bytes) 2024-09-18T01:10:13.714 INFO:teuthology.orchestra.run.smithi120.stdout:4626 write 0x770e98e thru 0x7716557 (0x7bca bytes) 2024-09-18T01:10:13.719 INFO:teuthology.orchestra.run.smithi120.stdout:4627 read 0xa101695 thru 0xa10ef87 (0xd8f3 bytes) 2024-09-18T01:10:13.719 INFO:teuthology.orchestra.run.smithi120.stdout:4628 writesame 0xa3fc6e2 thru 0xa3fdb9e data_size 0x1(0x14bd bytes) 2024-09-18T01:10:13.723 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.723 INFO:teuthology.orchestra.run.smithi120.stdout:4631 writesame 0xa7166ca thru 0xa7174b9 data_size 0x8(0xdf0 bytes) 2024-09-18T01:10:13.727 INFO:teuthology.orchestra.run.smithi120.stdout:4632 write 0xc10de43 thru 0xc10fd0c (0x1eca bytes) 2024-09-18T01:10:13.730 INFO:teuthology.orchestra.run.smithi120.stdout:4633 punch from 0x46d169f to 0x46dcd7b, (0xb6dc bytes) 2024-09-18T01:10:13.731 INFO:teuthology.orchestra.run.smithi120.stdout:4634 punch from 0xc5987a7 to 0xc5a2be0, (0xa439 bytes) 2024-09-18T01:10:13.731 INFO:teuthology.orchestra.run.smithi120.stdout:4635 read 0xbaa6c8d thru 0xbaae0d4 (0x7448 bytes) 2024-09-18T01:10:13.731 INFO:teuthology.orchestra.run.smithi120.stdout:4636 punch from 0xe4e734 to 0xe5bf2d, (0xd7f9 bytes) 2024-09-18T01:10:13.731 INFO:teuthology.orchestra.run.smithi120.stdout:4637 writesame 0x3a9c581 thru 0x3aa0d2d data_size 0x1(0x47ad bytes) 2024-09-18T01:10:13.741 INFO:teuthology.orchestra.run.smithi120.stdout:4639 punch from 0x253fe83 to 0x25438ab, (0x3a28 bytes) 2024-09-18T01:10:13.741 INFO:teuthology.orchestra.run.smithi120.stdout:4640 write 0x554275c thru 0x5548e9b (0x6740 bytes) 2024-09-18T01:10:13.745 INFO:teuthology.orchestra.run.smithi120.stdout:4641 read 0x90ad99 thru 0x912159 (0x73c1 bytes) 2024-09-18T01:10:13.745 INFO:teuthology.orchestra.run.smithi120.stdout:4643 write 0xa680efb thru 0xa6834c5 (0x25cb bytes) 2024-09-18T01:10:13.748 INFO:teuthology.orchestra.run.smithi120.stdout:4645 write 0x67ab2b3 thru 0x67acf21 (0x1c6f bytes) 2024-09-18T01:10:13.749 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:13.748+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:13.751 INFO:teuthology.orchestra.run.smithi120.stdout:4646 write 0x9ef4d88 thru 0x9effac8 (0xad41 bytes) 2024-09-18T01:10:13.755 INFO:teuthology.orchestra.run.smithi120.stdout:4647 punch from 0xbefbc4e to 0xbf021a3, (0x6555 bytes) 2024-09-18T01:10:13.755 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.759 INFO:teuthology.orchestra.run.smithi120.stdout:4649 trunc from 0xc89d9a6 to 0x84f8721 2024-09-18T01:10:13.771 INFO:teuthology.orchestra.run.smithi120.stdout:4650 write 0x7aabe61 thru 0x7ab31db (0x737b bytes) 2024-09-18T01:10:13.774 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.775 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.775 INFO:teuthology.orchestra.run.smithi120.stdout:4656 write 0x4629d96 thru 0x462e8f9 (0x4b64 bytes) 2024-09-18T01:10:13.779 INFO:teuthology.orchestra.run.smithi120.stdout:4658 read 0x7f20c59 thru 0x7f22795 (0x1b3d bytes) 2024-09-18T01:10:13.785 INFO:teuthology.orchestra.run.smithi120.stdout:4659 write 0x874cd00 thru 0x87571ec (0xa4ed bytes) 2024-09-18T01:10:13.790 INFO:teuthology.orchestra.run.smithi120.stdout:4661 read 0x10eafc2 thru 0x10efcf4 (0x4d33 bytes) 2024-09-18T01:10:13.790 INFO:teuthology.orchestra.run.smithi120.stdout:4662 punch from 0x3b2b7ed to 0x3b37386, (0xbb99 bytes) 2024-09-18T01:10:13.790 INFO:teuthology.orchestra.run.smithi120.stdout:4664 read 0x78f4e8b thru 0x78ff9d6 (0xab4c bytes) 2024-09-18T01:10:13.790 INFO:teuthology.orchestra.run.smithi120.stdout:4668 write 0x122e1d3 thru 0x123113b (0x2f69 bytes) 2024-09-18T01:10:13.793 INFO:teuthology.orchestra.run.smithi120.stdout:4671 punch from 0x1e7acbd to 0x1e7b597, (0x8da bytes) 2024-09-18T01:10:13.793 INFO:teuthology.orchestra.run.smithi120.stdout:4674 read 0x157f65b thru 0x158f275 (0xfc1b bytes) 2024-09-18T01:10:13.793 INFO:teuthology.orchestra.run.smithi120.stdout:4675 write 0x2942a07 thru 0x294d0fc (0xa6f6 bytes) 2024-09-18T01:10:13.796 INFO:teuthology.orchestra.run.smithi120.stdout:4676 writesame 0xa77ba5 thru 0xa86f25 data_size 0xb(0xf381 bytes) 2024-09-18T01:10:13.803 INFO:teuthology.orchestra.run.smithi120.stdout:4677 write 0x372c0f thru 0x37e0a6 (0xb498 bytes) 2024-09-18T01:10:13.805 INFO:teuthology.orchestra.run.smithi120.stdout:4678 punch from 0x657cdba to 0x657f3c3, (0x2609 bytes) 2024-09-18T01:10:13.813 INFO:teuthology.orchestra.run.smithi120.stdout:4679 write 0x9e9a6dc thru 0x9ea9259 (0xeb7e bytes) 2024-09-18T01:10:13.817 INFO:teuthology.orchestra.run.smithi120.stdout:4680 read 0x5070e29 thru 0x507655a (0x5732 bytes) 2024-09-18T01:10:13.817 INFO:teuthology.orchestra.run.smithi120.stdout:4681 writesame 0x48db35d thru 0x48e7f60 data_size 0x16(0xcc04 bytes) 2024-09-18T01:10:13.824 INFO:teuthology.orchestra.run.smithi120.stdout:4682 read 0x873cb05 thru 0x8742b59 (0x6055 bytes) 2024-09-18T01:10:13.825 INFO:teuthology.orchestra.run.smithi120.stdout:4683 read 0x2ca3a33 thru 0x2caa423 (0x69f1 bytes) 2024-09-18T01:10:13.825 INFO:teuthology.orchestra.run.smithi120.stdout:4684 read 0x9487b56 thru 0x9497004 (0xf4af bytes) 2024-09-18T01:10:13.825 INFO:teuthology.orchestra.run.smithi120.stdout:4685 write 0x45993b thru 0x463a37 (0xa0fd bytes) 2024-09-18T01:10:13.828 INFO:teuthology.orchestra.run.smithi120.stdout:4686 punch from 0x411c279 to 0x411e4ea, (0x2271 bytes) 2024-09-18T01:10:13.828 INFO:teuthology.orchestra.run.smithi120.stdout:4688 read 0x7860871 thru 0x7867e91 (0x7621 bytes) 2024-09-18T01:10:13.828 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:13.827+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:13.839 INFO:teuthology.orchestra.run.smithi120.stdout:4689 write 0xdf9bc6e thru 0xdfa9713 (0xdaa6 bytes) 2024-09-18T01:10:13.843 INFO:teuthology.orchestra.run.smithi120.stdout:4691 read 0xc39fea1 thru 0xc3ae0c9 (0xe229 bytes) 2024-09-18T01:10:13.858 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:13.858 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:13.858 INFO:teuthology.orchestra.run.smithi120.stdout:4692 trunc from 0xdfa9714 to 0x120d565 2024-09-18T01:10:13.860 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:13.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:13.860 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:10:13.860 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:10:13.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:13.883 INFO:teuthology.orchestra.run.smithi120.stdout:4693 writesame 0x15d0366 thru 0x15d88d8 data_size 0x1(0x8573 bytes) 2024-09-18T01:10:13.894 INFO:teuthology.orchestra.run.smithi120.stdout:4694 write 0x60cb144 thru 0x60d0151 (0x500e bytes) 2024-09-18T01:10:13.898 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.898 INFO:teuthology.orchestra.run.smithi120.stdout:4696 write 0x4dc220b thru 0x4dc6b21 (0x4917 bytes) 2024-09-18T01:10:13.903 INFO:teuthology.orchestra.run.smithi120.stdout:4697 punch from 0x4a28958 to 0x4a2ef91, (0x6639 bytes) 2024-09-18T01:10:13.903 INFO:teuthology.orchestra.run.smithi120.stdout:4698 punch from 0x3dae78a to 0x3daea58, (0x2ce bytes) 2024-09-18T01:10:13.912 INFO:teuthology.orchestra.run.smithi120.stdout:4701 writesame 0x7d2e3cb thru 0x7d2ead0 data_size 0x1f(0x706 bytes) 2024-09-18T01:10:13.912 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.912 INFO:teuthology.orchestra.run.smithi120.stdout:4704 write 0x3775b72 thru 0x377c4d2 (0x6961 bytes) 2024-09-18T01:10:13.927 INFO:teuthology.orchestra.run.smithi120.stdout:4705 write 0xb04de55 thru 0xb05961e (0xb7ca bytes) 2024-09-18T01:10:13.928 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:13.927+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:13.931 INFO:teuthology.orchestra.run.smithi120.stdout:4708 write 0x20c6b80 thru 0x20cfde8 (0x9269 bytes) 2024-09-18T01:10:13.935 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:13.935 INFO:teuthology.orchestra.run.smithi120.stdout:4710 read 0x3e5c675 thru 0x3e5e0db (0x1a67 bytes) 2024-09-18T01:10:13.946 INFO:teuthology.orchestra.run.smithi120.stdout:4711 trunc from 0xb05961f to 0x1eaef52 2024-09-18T01:10:13.962 INFO:teuthology.orchestra.run.smithi120.stdout:4712 read 0x17d579e thru 0x17dddc0 (0x8623 bytes) 2024-09-18T01:10:13.969 INFO:teuthology.orchestra.run.smithi120.stdout:4713 write 0x3000ded thru 0x300ec2b (0xde3f bytes) 2024-09-18T01:10:13.973 INFO:teuthology.orchestra.run.smithi120.stdout:4714 read 0x295562b thru 0x295c04c (0x6a22 bytes) 2024-09-18T01:10:13.973 INFO:teuthology.orchestra.run.smithi120.stdout:4715 read 0x29061cf thru 0x290972c (0x355e bytes) 2024-09-18T01:10:13.980 INFO:teuthology.orchestra.run.smithi120.stdout:4717 trunc from 0x300ec2c to 0x8b30ce2 2024-09-18T01:10:13.995 INFO:teuthology.orchestra.run.smithi120.stdout:4718 write 0xae47bd6 thru 0xae56de9 (0xf214 bytes) 2024-09-18T01:10:13.998 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.006 INFO:teuthology.orchestra.run.smithi120.stdout:4720 trunc from 0xae56dea to 0x47fc1cf 2024-09-18T01:10:14.017 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.017 INFO:teuthology.orchestra.run.smithi120.stdout:4722 read 0x1eb9b49 thru 0x1ebc016 (0x24ce bytes) 2024-09-18T01:10:14.017 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.017 INFO:teuthology.orchestra.run.smithi120.stdout:4724 read 0x13cddb thru 0x141dd9 (0x4fff bytes) 2024-09-18T01:10:14.018 INFO:teuthology.orchestra.run.smithi120.stdout:4726 punch from 0x2a026ad to 0x2a1124d, (0xeba0 bytes) 2024-09-18T01:10:14.018 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.018 INFO:teuthology.orchestra.run.smithi120.stdout:4728 write 0x3462a6f thru 0x347111d (0xe6af bytes) 2024-09-18T01:10:14.020 INFO:teuthology.orchestra.run.smithi120.stdout:4730 writesame 0x3c9e743 thru 0x3ca9d96 data_size 0x4(0xb654 bytes) 2024-09-18T01:10:14.023 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.023 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.029 INFO:teuthology.orchestra.run.smithi120.stdout:4731 write 0x51113f thru 0x51ecdf (0xdba1 bytes) 2024-09-18T01:10:14.029 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:14.028+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:14.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:14.040 INFO:teuthology.orchestra.run.smithi120.stdout:4732 write 0x662b13a thru 0x66316e1 (0x65a8 bytes) 2024-09-18T01:10:14.043 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.050 INFO:teuthology.orchestra.run.smithi120.stdout:4735 trunc from 0x66316e2 to 0xcaee4b 2024-09-18T01:10:14.091 INFO:teuthology.orchestra.run.smithi120.stdout:4736 writesame 0x57ccd5a thru 0x57ccf7c data_size 0x1(0x223 bytes) 2024-09-18T01:10:14.092 INFO:teuthology.orchestra.run.smithi120.stdout:4737 writesame 0x29569df thru 0x2962190 data_size 0x7(0xb7b2 bytes) 2024-09-18T01:10:14.097 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.102 INFO:teuthology.orchestra.run.smithi120.stdout:4739 trunc from 0x57ccf7d to 0x154a47a 2024-09-18T01:10:14.113 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.121 INFO:teuthology.orchestra.run.smithi120.stdout:4742 write 0x33487d6 thru 0x3358609 (0xfe34 bytes) 2024-09-18T01:10:14.130 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:14.129+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:14.131 INFO:teuthology.orchestra.run.smithi120.stdout:4743 writesame 0x3f8d64c thru 0x3f983d9 data_size 0xf(0xad8e bytes) 2024-09-18T01:10:14.131 INFO:teuthology.orchestra.run.smithi120.stdout:4744 read 0x2d98dbe thru 0x2da8a6c (0xfcaf bytes) 2024-09-18T01:10:14.131 INFO:teuthology.orchestra.run.smithi120.stdout:4745 punch from 0x3b43cf2 to 0x3b4f062, (0xb370 bytes) 2024-09-18T01:10:14.131 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.142 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.142 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.149 INFO:teuthology.orchestra.run.smithi120.stdout:4748 write 0xdea675b thru 0xdeb5a5e (0xf304 bytes) 2024-09-18T01:10:14.152 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:14.165 INFO:teuthology.orchestra.run.smithi120.stdout:4750 trunc from 0xdeb5a5f to 0x2efa8b0 2024-09-18T01:10:14.169 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:10:14.169 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 178, 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:10:12.628065+0000", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:09:33.643251+0000", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5799936, 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:10:14.170 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-18T01:10:14.171 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:14.172 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:10:14.173 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:14.174 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:14.175 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:14.176 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:10:14.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:14.178 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:14.179 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:14.180 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:10:14.181 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:14.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:14.183 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:10:14.184 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:10:14.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:10:14.186 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "178", 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:14.187 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 79, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 178, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:14.188 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.189 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:14.190 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.192 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 166, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:10:14.193 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.194 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:10:14.197 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.198 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 143, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:10:14.199 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.200 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.201 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.202 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.203 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.204 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 162, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.205 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.207 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.210 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:10:14.211 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 113, 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:10:14.212 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.213 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 163, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:10:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:10:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 140, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:10:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:10:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:10:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 163, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 164, 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:10:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1a", 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-18T01:10:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:10:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 75, 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 77, 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 79, 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 79, 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:10:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:10:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:10:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:10:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:10:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:14.253 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:14.254 INFO:teuthology.orchestra.run.smithi120.stdout:4752 punch from 0x8921f9 to 0x8a0e2d, (0xec34 bytes) 2024-09-18T01:10:14.255 INFO:tasks.thrashosds.thrasher:j is {'epoch': 178, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:10:12.628065+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:09:33.643251+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 79, 'snap_epoch': 178, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 162, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 140, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 75, 'length': 1}, {'begin': 77, 'length': 1}, {'begin': 79, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 79, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:10:14.255 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1a 2024-09-18T01:10:14.256 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.1a 2024-09-18T01:10:14.257 INFO:teuthology.orchestra.run.smithi120.stdout:4753 write 0x93fa962 thru 0x93faadd (0x17c bytes) 2024-09-18T01:10:14.257 INFO:teuthology.orchestra.run.smithi120.stdout:4754 read 0x474e86e thru 0x4751dc2 (0x3555 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4755 read 0x843f76d thru 0x8443ef6 (0x478a bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4756 trunc from 0x93faade to 0xb9523d9 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4757 read 0xd8f54c thru 0xd92452 (0x2f07 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4758 read 0x3f0049b thru 0x3f0bd2b (0xb891 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4760 writesame 0xcd70066 thru 0xcd7919d data_size 0xc(0x9138 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4761 read 0x69a013d thru 0x69af6a4 (0xf568 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4762 write 0x915cd51 thru 0x915f3c5 (0x2675 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4766 read 0x61363a3 thru 0x613a0aa (0x3d08 bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4767 read 0x491d4a1 thru 0x4921b0f (0x466f bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4768 write 0x882d62a thru 0x8836f38 (0x990f bytes) 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.258 INFO:teuthology.orchestra.run.smithi120.stdout:4770 trunc from 0xcd7919e to 0x7d91451 2024-09-18T01:10:14.259 INFO:teuthology.orchestra.run.smithi120.stdout:4771 read 0x3d1a185 thru 0x3d2730d (0xd189 bytes) 2024-09-18T01:10:14.259 INFO:teuthology.orchestra.run.smithi120.stdout:4774 writesame 0x742d2f1 thru 0x7434b13 data_size 0x1(0x7823 bytes) 2024-09-18T01:10:14.259 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:14.253+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.261 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:4776 punch from 0x4f390f2 to 0x4f39a1e, (0x92c bytes) 2024-09-18T01:10:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:4777 read 0x46f680b thru 0x4704bbb (0xe3b1 bytes) 2024-09-18T01:10:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:4778 read 0x10ea046 thru 0x10ea6b8 (0x673 bytes) 2024-09-18T01:10:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:4779 writesame 0x18f2625 thru 0x18f2f99 data_size 0x3(0x975 bytes) 2024-09-18T01:10:14.268 INFO:teuthology.orchestra.run.smithi120.stdout:4781 trunc from 0x7d91451 to 0x998b120 2024-09-18T01:10:14.271 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:14.283 INFO:teuthology.orchestra.run.smithi120.stdout:4782 write 0xc385f18 thru 0xc38c3bb (0x64a4 bytes) 2024-09-18T01:10:14.286 INFO:teuthology.orchestra.run.smithi120.stdout:4783 punch from 0x45fb5ef to 0x45fbc19, (0x62a bytes) 2024-09-18T01:10:14.294 INFO:teuthology.orchestra.run.smithi120.stdout:4784 write 0xe9dc4f9 thru 0xe9df6a6 (0x31ae bytes) 2024-09-18T01:10:14.299 INFO:teuthology.orchestra.run.smithi120.stdout:4785 read 0x70f1d06 thru 0x70f3205 (0x1500 bytes) 2024-09-18T01:10:14.299 INFO:teuthology.orchestra.run.smithi120.stdout:4786 write 0xb36104a thru 0xb3677b8 (0x676f bytes) 2024-09-18T01:10:14.302 INFO:teuthology.orchestra.run.smithi120.stdout:4787 writesame 0xde63bd7 thru 0xde67071 data_size 0x3(0x349b bytes) 2024-09-18T01:10:14.307 INFO:teuthology.orchestra.run.smithi120.stdout:4788 writesame 0x28b679a thru 0x28ba0bb data_size 0x2(0x3922 bytes) 2024-09-18T01:10:14.314 INFO:teuthology.orchestra.run.smithi120.stdout:4789 punch from 0x88a4ad9 to 0x88a88b0, (0x3dd7 bytes) 2024-09-18T01:10:14.314 INFO:teuthology.orchestra.run.smithi120.stdout:4790 punch from 0x3563b16 to 0x356c11a, (0x8604 bytes) 2024-09-18T01:10:14.314 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.314 INFO:teuthology.orchestra.run.smithi120.stdout:4793 write 0x5632987 thru 0x5632b7e (0x1f8 bytes) 2024-09-18T01:10:14.317 INFO:teuthology.orchestra.run.smithi120.stdout:4795 read 0x6cd4b48 thru 0x6ce0ffb (0xc4b4 bytes) 2024-09-18T01:10:14.317 INFO:teuthology.orchestra.run.smithi120.stdout:4796 read 0x9e4d80b thru 0x9e51bec (0x43e2 bytes) 2024-09-18T01:10:14.317 INFO:teuthology.orchestra.run.smithi120.stdout:4797 read 0xc79971f thru 0xc7a8bb1 (0xf493 bytes) 2024-09-18T01:10:14.317 INFO:teuthology.orchestra.run.smithi120.stdout:4798 read 0xcda936 thru 0xce1171 (0x683c bytes) 2024-09-18T01:10:14.318 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:14.318 INFO:teuthology.orchestra.run.smithi120.stdout:4800 write 0x44b9d70 thru 0x44c8c2e (0xeebf bytes) 2024-09-18T01:10:14.321 INFO:teuthology.orchestra.run.smithi120.stdout:4801 read 0x611908d thru 0x6121274 (0x81e8 bytes) 2024-09-18T01:10:14.330 INFO:teuthology.orchestra.run.smithi120.stdout:4803 trunc from 0xe9df6a7 to 0x749d35c 2024-09-18T01:10:14.343 INFO:teuthology.orchestra.run.smithi120.stdout:4804 write 0x4dec1e5 thru 0x4deee0b (0x2c27 bytes) 2024-09-18T01:10:14.347 INFO:teuthology.orchestra.run.smithi120.stdout:4805 read 0x5fd27f9 thru 0x5fdeff0 (0xc7f8 bytes) 2024-09-18T01:10:14.347 INFO:teuthology.orchestra.run.smithi120.stdout:4806 write 0xa49349 thru 0xa4c925 (0x35dd bytes) 2024-09-18T01:10:14.348 INFO:teuthology.orchestra.run.smithi120.stdout:4807 writesame 0x3a66015 thru 0x3a684a0 data_size 0x2(0x248c bytes) 2024-09-18T01:10:14.355 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:14.354+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:14.358 INFO:teuthology.orchestra.run.smithi120.stdout:4808 trunc from 0x749d35c to 0xbafa2e6 2024-09-18T01:10:14.363 INFO:teuthology.orchestra.run.smithi120.stdout:4811 punch from 0xb8d0e35 to 0xb8dae6b, (0xa036 bytes) 2024-09-18T01:10:14.363 INFO:teuthology.orchestra.run.smithi120.stdout:4812 clone 39 order 20 su 8192 sc 8 2024-09-18T01:10:14.418 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.418 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.426 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:14.467 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:14.466+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:14.557 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:14.556+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:clear 3.1a pg_upmap_items mapping 2024-09-18T01:10:14.572 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.572 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:14.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:10:14.693 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:14.692+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:14.726 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.726 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.735 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:14.759 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:14.758+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:14.842 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.842 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:14.897 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:14.896+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:14.948 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:14.958 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:14.958 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:14.961 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:14.960+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:14.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:15.062 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:15.061+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:15.110 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:15.111 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:15.120 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:15.164 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:15.163+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:15.222 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:15.222 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:15.231 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-18T01:10:15.248+0000 7f17ec9bf640 0 --2- 172.21.15.120:0/3148865506 >> v2:172.21.15.120:3300/0 conn(0x7f17e80b2c40 0x7f17e81431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T01:10:15.264 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:15.263+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:15.272 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:15.272 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:15.289 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:15.289 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:15.305 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:15.321 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:15.322 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:15.338 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:15.339 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:15.353 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:15.353 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:15.353 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:15.355 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:15.355 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:15.362 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:15.374 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:15.374 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:15.374 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:15.373+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:15.391 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:15.391 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:15.465 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:15.464+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.565254122999999, 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:15.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.565046091999999, 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:15.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:15.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.551009517000001, 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:15.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550986169000002, 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:15.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:15.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550964648000001, 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550943928999999, 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:15.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550923152999999, 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:15.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:15.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550903007999999, 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:15.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:15.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:15.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550882632, 2024-09-18T01:10:15.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:15.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550861904000001, 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:15.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550837445999999, 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:15.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.550816969, 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:15.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:15.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.542040773, 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:15.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:15.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:15.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.542015772999999, 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:15.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:15.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541978618000002, 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:15.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:15.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541951935, 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:15.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:15.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541932035999999, 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:15.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541912829000001, 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:15.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541893815999998, 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:15.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.541875543, 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:15.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:15.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.574 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:15.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:15.576 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:10:15.576 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:15.574+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:15.676 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:15.675+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:15.727 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:15.727 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:15.727 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:15.727 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:15.735 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:15.777 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:15.776+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:15.837 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:15.837 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:15.837 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:15.838 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:15.838 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:15.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:15.878 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:15.877+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.949 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.286266486999999, 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.950 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:15.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:15.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.286212261999999, 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.955 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:15.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.960 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.286179804, 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.961 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:15.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.286048541, 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:15.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.285578555000001, 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:15.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:15.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:15.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.284415955, 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:15.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:15.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.284308672999998, 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:15.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.284227014000002, 2024-09-18T01:10:15.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:15.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:15.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:15.992 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.283711625999999, 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.993 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:15.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:15.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.282892388, 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:15.998 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:16.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:16.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.709627302000001, 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:16.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:16.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.542768410000001, 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:16.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:16.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:16.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:16.016 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.514923021999998, 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:16.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.514902722999999, 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:16.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:16.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:16.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.514882460000003, 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:16.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:16.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.514862606999998, 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:16.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:16.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.041 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.514843450000001, 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.042 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:16.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.130853023999997, 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.048 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:16.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:16.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.070894160999998, 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.054 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:16.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:16.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2863907149999996, 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:16.060 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:16.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:16.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.065 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:16.066 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:16.067 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:16.065+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:16.168 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:16.167+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:16.170 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:16.170 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:16.170 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:16.180 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:16.272 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:16.271+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:16.330 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:16.330 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:16.330 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:16.330 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:16.330 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:16.338 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:16.380 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:16.379+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:16.471 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:16.470+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:16.482 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820593154999999, 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820573959000001, 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:10:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:10:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820555598, 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:10:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820537494, 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:10:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:10:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820518896999999, 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:10:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820495897000001, 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:10:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:10:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820477224999999, 2024-09-18T01:10:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:10:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:10:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820457222, 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820438982000001, 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:10:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:10:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820420567999999, 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.820398540999999, 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:10:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816146762000001, 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816136309999999, 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:10:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:10:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816124619, 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:10:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816111321999999, 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:10:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816095251, 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816078593, 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:10:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:10:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:10:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816059298000001, 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:10:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816031068000001, 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:10:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.816009943999999, 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:10:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:10:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:16.581 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:16.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:16.583 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:16.582+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:16.683 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:16.683+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:16.700 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:16.701 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:16.701 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:16.701 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:16.710 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:16.785 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:16.784+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:16.820 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:16.820 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:16.820 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:16.829 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:16.886 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:16.885+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:16.933 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 139.20875912400001, 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:16.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:16.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:16.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177732601999999, 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.941 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:16.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:16.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177723023999999, 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:16.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177708998, 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:16.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:16.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177680225, 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:16.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177652529, 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:16.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:16.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177618080999999, 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:16.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177588235000002, 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:16.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:16.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:16.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177566043999999, 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:16.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177529138000001, 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:16.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:16.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:16.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:16.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177508023000001, 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.991 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:16.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.177477498000002, 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:16.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:16.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:16.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:16.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:17.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171997244, 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:17.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171943384999999, 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:17.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171865962999998, 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:17.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:17.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:17.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171835849000001, 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:17.018 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171785930999999, 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:17.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:17.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:17.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171752344000002, 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:17.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.17172781, 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:17.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.171542528, 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.040 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.045 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:17.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:17.047 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:17.045+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:17.147 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:17.147+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:17.190 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:17.191 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:17.191 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:17.191 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:17.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:17.249 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:17.248+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:17.302 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:17.302 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:17.302 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:17.310 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:17.350 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:17.349+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:17.403 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:17.450 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:17.449+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.854667698, 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853517683, 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:17.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:17.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853498372000001, 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:17.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.472 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:17.472 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853470626, 2024-09-18T01:10:17.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:17.472 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:17.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853450773999999, 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:17.477 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:17.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.85343091, 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853412704, 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853392657000001, 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:17.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:17.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853374449, 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:17.498 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:17.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853356337000001, 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:17.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853338419, 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:17.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.853320399000001, 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:17.512 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:17.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:17.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852767443000001, 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:17.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852748720000001, 2024-09-18T01:10:17.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:17.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:17.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:17.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852718936999999, 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:17.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852654849, 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:17.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:17.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852470547999999, 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:17.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:17.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852452394, 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:17.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852424022000001, 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:17.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:17.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:17.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.852402095999999, 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:17.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:17.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:17.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:17.559 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:17.560 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:17.560 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:17.561 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:17.561 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:17.562 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:17.562 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:17.559+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:17.563 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:17.563 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:17.577 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:17.578 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:17.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:10:17.594 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:17.595 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:17.611 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:17.611 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:17.643 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:17.643 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:17.661 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:17.660+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:17.668 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:17.668 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:17.707 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:17.708 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:17.717 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:17.762 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:17.761+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:17.822 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:17.822 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:17.831 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:17.862 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:17.862+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:17.934 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:17.934 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:17.942 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:17.987 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:17.986+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:18.047 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.047 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:18.097 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:18.096+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:18.165 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:18.164+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:18.201 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.201 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:18.266 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:18.265+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:18.353 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.354 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.362 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:18.407 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:18.406+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:18.468 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:18.467+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:18.469 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.469 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:18.569 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:18.568+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:18.576 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:10:18.655 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.655 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.670 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:18.670 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:18.668+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:18.771 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:18.770+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:18.853 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:18.853 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:18.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:18.872 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:18.871+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:18.974 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:18.973+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:19.006 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.007 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.015 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:19.099 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:19.098+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:19.157 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.157 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.165 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:19.176 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:19.175+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:19.277 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:19.276+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:19.308 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.308 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.316 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:19.378 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:19.377+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:19.459 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.459 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.467 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:19.511 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:19.510+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.579 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:19.579 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:19.579 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-18T01:10:19.580 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-18T01:10:19.580 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:19.578+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:19.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:19.680 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:19.681 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:19.679+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-18T01:10:19.739 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:19.740 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:19.756 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:19.781 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:19.780+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:19.842 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:19.861 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:19.861 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:19.877 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:19.878 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:19.882 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:19.881+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:19.887 INFO:teuthology.orchestra.run.smithi120.stdout:4814 read 0x28b55cf thru 0x28c0d77 (0xb7a9 bytes) 2024-09-18T01:10:19.888 INFO:teuthology.orchestra.run.smithi120.stdout:4815 write 0x4767fb5 thru 0x476ebea (0x6c36 bytes) 2024-09-18T01:10:19.896 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:19.896 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:19.898 INFO:teuthology.orchestra.run.smithi120.stdout:4816 write 0x1eb3dd7 thru 0x1ebb2d9 (0x7503 bytes) 2024-09-18T01:10:19.903 INFO:teuthology.orchestra.run.smithi120.stdout:4818 writesame 0x46b0fc6 thru 0x46beb24 data_size 0x1(0xdb5f bytes) 2024-09-18T01:10:19.911 INFO:tasks.ceph.mon.a.smithi120.stderr:2024-09-18T01:10:19.910+0000 7f77b87d3640 -1 mon.a@0(leader).osd e184 definitely_dead 0 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:238019 3.b 3:d73f7061:::rbd_data.117677c7714b.0000000000000040:head [write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:19.903394+0000", 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.012354619000000001, 2024-09-18T01:10:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.012378894999999999, 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 238019 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903394+0000", 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903394+0000", 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903395+0000", 2024-09-18T01:10:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.398e-06 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903401+0000", 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.699e-06 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903412+0000", 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0971e-05 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903419+0000", 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903466+0000", 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7539e-05 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.903552+0000", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5347999999999997e-05 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.904130+0000", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057774900000000001 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:19.907251+0000", 2024-09-18T01:10:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031219099999999999 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-09-18T01:10:19.922 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:19.923 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:19.923 INFO:teuthology.orchestra.run.smithi120.stdout:4820 write 0x76db532 thru 0x76e9efe (0xe9cd bytes) 2024-09-18T01:10:19.923 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:19.923 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:19.924 INFO:teuthology.orchestra.run.smithi120.stdout:4821 write 0x86a2ad2 thru 0x86a9a6c (0x6f9b bytes) 2024-09-18T01:10:19.927 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:19.931 INFO:teuthology.orchestra.run.smithi120.stdout:4822 read 0x9fcf5fd thru 0x9fdb5f2 (0xbff6 bytes) 2024-09-18T01:10:19.931 INFO:teuthology.orchestra.run.smithi120.stdout:4823 read 0xa1f71f5 thru 0xa1fd310 (0x611c bytes) 2024-09-18T01:10:19.937 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:19.941 INFO:teuthology.orchestra.run.smithi120.stdout:4824 write 0xee2e9de thru 0xee3c79f (0xddc2 bytes) 2024-09-18T01:10:19.954 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:19.955 INFO:teuthology.orchestra.run.smithi120.stdout:4827 writesame 0x37a597 thru 0x37dd96 data_size 0x38(0x3800 bytes) 2024-09-18T01:10:19.955 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:19.955 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:19.957 INFO:teuthology.orchestra.run.smithi120.stdout:4829 write 0xc123303 thru 0xc12abe2 (0x78e0 bytes) 2024-09-18T01:10:19.965 INFO:teuthology.orchestra.run.smithi120.stdout:4831 write 0xdf1c8fc thru 0xdf26d64 (0xa469 bytes) 2024-09-18T01:10:19.975 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:19.975 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:19.979 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:19.979 INFO:teuthology.orchestra.run.smithi120.stdout:4833 punch from 0x3539dc1 to 0x35411e5, (0x7424 bytes) 2024-09-18T01:10:19.979 INFO:teuthology.orchestra.run.smithi120.stdout:4835 write 0x2c3c5cd thru 0x2c45cca (0x96fe bytes) 2024-09-18T01:10:19.982 INFO:teuthology.orchestra.run.smithi120.stdout:4836 write 0x59620b0 thru 0x596bced (0x9c3e bytes) 2024-09-18T01:10:19.982 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:19.981+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:19.991 INFO:teuthology.orchestra.run.smithi120.stdout:4837 writesame 0xa9a2dc1 thru 0xa9abadb data_size 0x1(0x8d1b bytes) 2024-09-18T01:10:20.004 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.004 INFO:teuthology.orchestra.run.smithi120.stdout:4839 read 0x207c404 thru 0x2082b8f (0x678c bytes) 2024-09-18T01:10:20.005 INFO:teuthology.orchestra.run.smithi120.stdout:4842 read 0x5eb33d3 thru 0x5ec0126 (0xcd54 bytes) 2024-09-18T01:10:20.005 INFO:teuthology.orchestra.run.smithi120.stdout:4844 write 0x10d17b6 thru 0x10d7e90 (0x66db bytes) 2024-09-18T01:10:20.013 INFO:teuthology.orchestra.run.smithi120.stdout:4846 read 0x64ebdcb thru 0x64ee3f8 (0x262e bytes) 2024-09-18T01:10:20.013 INFO:teuthology.orchestra.run.smithi120.stdout:4847 trunc from 0xee3c7a0 to 0xec0d10a 2024-09-18T01:10:20.023 INFO:teuthology.orchestra.run.smithi120.stdout:4849 write 0xb0a60d thru 0xb15eb1 (0xb8a5 bytes) 2024-09-18T01:10:20.030 INFO:teuthology.orchestra.run.smithi120.stdout:4852 read 0xd358cf2 thru 0xd361bca (0x8ed9 bytes) 2024-09-18T01:10:20.030 INFO:teuthology.orchestra.run.smithi120.stdout:4857 read 0x9d4a86b thru 0x9d4ccc3 (0x2459 bytes) 2024-09-18T01:10:20.030 INFO:teuthology.orchestra.run.smithi120.stdout:4858 write 0x38916ad thru 0x3895370 (0x3cc4 bytes) 2024-09-18T01:10:20.032 INFO:teuthology.orchestra.run.smithi120.stdout:4860 read 0x1a31b thru 0x1cdef (0x2ad5 bytes) 2024-09-18T01:10:20.033 INFO:teuthology.orchestra.run.smithi120.stdout:4861 read 0x1e19031 thru 0x1e1d773 (0x4743 bytes) 2024-09-18T01:10:20.033 INFO:teuthology.orchestra.run.smithi120.stdout:4862 read 0xc1d7c38 thru 0xc1e749c (0xf865 bytes) 2024-09-18T01:10:20.034 INFO:teuthology.orchestra.run.smithi120.stdout:4863 punch from 0x7e0854a to 0x7e085b6, (0x6c bytes) 2024-09-18T01:10:20.034 INFO:teuthology.orchestra.run.smithi120.stdout:4864 write 0xc34902 thru 0xc41834 (0xcf33 bytes) 2024-09-18T01:10:20.041 INFO:teuthology.orchestra.run.smithi120.stdout:4865 punch from 0x1f6e295 to 0x1f7af66, (0xccd1 bytes) 2024-09-18T01:10:20.041 INFO:teuthology.orchestra.run.smithi120.stdout:4866 punch from 0x12e60cf to 0x12f0215, (0xa146 bytes) 2024-09-18T01:10:20.041 INFO:teuthology.orchestra.run.smithi120.stdout:4867 write 0xc7954cf thru 0xc7a156d (0xc09f bytes) 2024-09-18T01:10:20.046 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:20.046 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:20.046 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:20.047 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:20.047 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:20.053 INFO:teuthology.orchestra.run.smithi120.stdout:4868 read 0x71647f0 thru 0x7164daf (0x5c0 bytes) 2024-09-18T01:10:20.053 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.053 INFO:teuthology.orchestra.run.smithi120.stdout:4871 punch from 0xd71116f to 0xd71a36d, (0x91fe bytes) 2024-09-18T01:10:20.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:20.064 INFO:teuthology.orchestra.run.smithi120.stdout:4872 trunc from 0xec0d10a to 0x5c8c5f2 2024-09-18T01:10:20.098 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.098 INFO:teuthology.orchestra.run.smithi120.stdout:4874 punch from 0x537ff62 to 0x538b413, (0xb4b1 bytes) 2024-09-18T01:10:20.098 INFO:teuthology.orchestra.run.smithi120.stdout:4875 read 0x35608a6 thru 0x356de61 (0xd5bc bytes) 2024-09-18T01:10:20.099 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:20.098+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:20.099 INFO:teuthology.orchestra.run.smithi120.stdout:4876 read 0x396f056 thru 0x397b09f (0xc04a bytes) 2024-09-18T01:10:20.100 INFO:teuthology.orchestra.run.smithi120.stdout:4878 write 0x4d6707d thru 0x4d75798 (0xe71c bytes) 2024-09-18T01:10:20.104 INFO:teuthology.orchestra.run.smithi120.stdout:4879 read 0x17cfa4 thru 0x18b68a (0xe6e7 bytes) 2024-09-18T01:10:20.111 INFO:teuthology.orchestra.run.smithi120.stdout:4880 trunc from 0x5c8c5f2 to 0xa9fc1d 2024-09-18T01:10:20.167 INFO:teuthology.orchestra.run.smithi120.stdout:4881 writesame 0x5336fb3 thru 0x533f788 data_size 0x1(0x87d6 bytes) 2024-09-18T01:10:20.172 INFO:teuthology.orchestra.run.smithi120.stdout:4882 trunc from 0x533f789 to 0xd2b8b5 2024-09-18T01:10:20.185 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:20.184+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527794451999998, 2024-09-18T01:10:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:10:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527775255999998, 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:10:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527756895, 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:10:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527738791000001, 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:10:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527720194, 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:10:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527697194000002, 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527678521999999, 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:20.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527658518999999, 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:20.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:10:20.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527640279, 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:20.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:10:20.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:10:20.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527621865, 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:10:20.235 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:10:20.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:10:20.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.240 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.527599838, 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:10:20.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:10:20.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:10:20.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523348059, 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.248 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:20.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:10:20.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523337606999998, 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:20.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:10:20.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:10:20.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523325916000001, 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:20.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:10:20.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523312618999999, 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:10:20.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:10:20.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523296548000001, 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:10:20.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:10:20.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523279890000001, 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.276 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:20.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:10:20.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523260595, 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:10:20.282 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:20.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:10:20.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:10:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523232364999998, 2024-09-18T01:10:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:20.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.523211240999998, 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.294 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:10:20.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.298 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:20.299 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:20.299 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:20.300 INFO:teuthology.orchestra.run.smithi120.stdout:4883 write 0x616d784 thru 0x6176c9a (0x9517 bytes) 2024-09-18T01:10:20.300 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:20.299+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:20.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:20.302 INFO:teuthology.orchestra.run.smithi120.stdout:4886 write 0x8684862 thru 0x868e598 (0x9d37 bytes) 2024-09-18T01:10:20.302 INFO:teuthology.orchestra.run.smithi120.stdout:4887 trunc from 0x868e599 to 0xaedfcc1 2024-09-18T01:10:20.302 INFO:teuthology.orchestra.run.smithi120.stdout:4888 writesame 0xd40df49 thru 0xd41c1d3 data_size 0x7(0xe28b bytes) 2024-09-18T01:10:20.302 INFO:teuthology.orchestra.run.smithi120.stdout:4889 writesame 0x2fda296 thru 0x2fe422f data_size 0x1f(0x9f9a bytes) 2024-09-18T01:10:20.302 INFO:teuthology.orchestra.run.smithi120.stdout:4892 trunc from 0xd41c1d4 to 0x19c6ce4 2024-09-18T01:10:20.303 INFO:teuthology.orchestra.run.smithi120.stdout:4893 write 0xde34b8d thru 0xde380df (0x3553 bytes) 2024-09-18T01:10:20.315 INFO:teuthology.orchestra.run.smithi120.stdout:4897 writesame 0x1ef2ca6 thru 0x1efb493 data_size 0x7f(0x87ee bytes) 2024-09-18T01:10:20.323 INFO:teuthology.orchestra.run.smithi120.stdout:4898 write 0x9441dae thru 0x944e421 (0xc674 bytes) 2024-09-18T01:10:20.337 INFO:teuthology.orchestra.run.smithi120.stdout:4899 write 0x9bfec22 thru 0x9bffdcb (0x11aa bytes) 2024-09-18T01:10:20.340 INFO:teuthology.orchestra.run.smithi120.stdout:4900 read 0x91be43c thru 0x91c08e3 (0x24a8 bytes) 2024-09-18T01:10:20.341 INFO:teuthology.orchestra.run.smithi120.stdout:4901 punch from 0x33a80f9 to 0x33a9007, (0xf0e bytes) 2024-09-18T01:10:20.353 INFO:teuthology.orchestra.run.smithi120.stdout:4902 trunc from 0xde380e0 to 0x3f26c15 2024-09-18T01:10:20.387 INFO:teuthology.orchestra.run.smithi120.stdout:4903 write 0xcbbc224 thru 0xcbc9bb3 (0xd990 bytes) 2024-09-18T01:10:20.400 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:20.399+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:20.403 INFO:teuthology.orchestra.run.smithi120.stdout:4904 punch from 0x8c00e3c to 0x8c0ee2b, (0xdfef bytes) 2024-09-18T01:10:20.403 INFO:teuthology.orchestra.run.smithi120.stdout:4905 punch from 0x47a0275 to 0x47a18ce, (0x1659 bytes) 2024-09-18T01:10:20.403 INFO:teuthology.orchestra.run.smithi120.stdout:4907 write 0x5b4477 thru 0x5c0290 (0xbe1a bytes) 2024-09-18T01:10:20.408 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.408 INFO:teuthology.orchestra.run.smithi120.stdout:4911 read 0x5552391 thru 0x5555773 (0x33e3 bytes) 2024-09-18T01:10:20.408 INFO:teuthology.orchestra.run.smithi120.stdout:4912 punch from 0x5b0c2a2 to 0x5b17d25, (0xba83 bytes) 2024-09-18T01:10:20.408 INFO:teuthology.orchestra.run.smithi120.stdout:4916 read 0x3d0808c thru 0x3d17596 (0xf50b bytes) 2024-09-18T01:10:20.409 INFO:teuthology.orchestra.run.smithi120.stdout:4917 read 0x91bdbbc thru 0x91c4bef (0x7034 bytes) 2024-09-18T01:10:20.409 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.409 INFO:teuthology.orchestra.run.smithi120.stdout:4920 write 0x2032657 thru 0x203a2b5 (0x7c5f bytes) 2024-09-18T01:10:20.419 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:20.419 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:20.419 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:20.419 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:20.427 INFO:teuthology.orchestra.run.smithi120.stdout:4921 write 0xd8f67a2 thru 0xd8fde49 (0x76a8 bytes) 2024-09-18T01:10:20.429 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:20.437 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.437 INFO:teuthology.orchestra.run.smithi120.stdout:4924 punch from 0x4d8a281 to 0x4d9066e, (0x63ed bytes) 2024-09-18T01:10:20.437 INFO:teuthology.orchestra.run.smithi120.stdout:4925 writesame 0x73d9469 thru 0x73df4d4 data_size 0x2c(0x606c bytes) 2024-09-18T01:10:20.446 INFO:teuthology.orchestra.run.smithi120.stdout:4927 write 0xc5877b3 thru 0xc58e643 (0x6e91 bytes) 2024-09-18T01:10:20.457 INFO:teuthology.orchestra.run.smithi120.stdout:4929 read 0x572034c thru 0x572f8c9 (0xf57e bytes) 2024-09-18T01:10:20.457 INFO:teuthology.orchestra.run.smithi120.stdout:4930 write 0x1efb083 thru 0x1f03a69 (0x89e7 bytes) 2024-09-18T01:10:20.467 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.467 INFO:teuthology.orchestra.run.smithi120.stdout:4932 read 0x9cecc7f thru 0x9cedffc (0x137e bytes) 2024-09-18T01:10:20.467 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.469 INFO:teuthology.orchestra.run.smithi120.stdout:4936 trunc from 0xd8fde4a to 0xba72d44 2024-09-18T01:10:20.493 INFO:teuthology.orchestra.run.smithi120.stdout:4937 write 0xdc0c379 thru 0xdc0fc85 (0x390d bytes) 2024-09-18T01:10:20.497 INFO:teuthology.orchestra.run.smithi120.stdout:4938 write 0x1318d3a thru 0x1324dee (0xc0b5 bytes) 2024-09-18T01:10:20.501 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:20.501+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:20.508 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.508 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.508 INFO:teuthology.orchestra.run.smithi120.stdout:4943 punch from 0x8e14d69 to 0x8e1c8d6, (0x7b6d bytes) 2024-09-18T01:10:20.508 INFO:teuthology.orchestra.run.smithi120.stdout:4944 write 0xd871512 thru 0xd87f434 (0xdf23 bytes) 2024-09-18T01:10:20.518 INFO:teuthology.orchestra.run.smithi120.stdout:4945 write 0x7ed4445 thru 0x7ee2a1c (0xe5d8 bytes) 2024-09-18T01:10:20.529 INFO:teuthology.orchestra.run.smithi120.stdout:4946 write 0x66f7d thru 0x74cf3 (0xdd77 bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4947 read 0x9a4c81c thru 0x9a4eb91 (0x2376 bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4948 punch from 0x412fc39 to 0x412ffe9, (0x3b0 bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4949 punch from 0x19333e8 to 0x193c1a6, (0x8dbe bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4950 punch from 0xd8031b5 to 0xd80a21c, (0x7067 bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4951 read 0xb985d38 thru 0xb99157f (0xb848 bytes) 2024-09-18T01:10:20.538 INFO:teuthology.orchestra.run.smithi120.stdout:4952 read 0xb94b67 thru 0xba09a6 (0xbe40 bytes) 2024-09-18T01:10:20.539 INFO:teuthology.orchestra.run.smithi120.stdout:4953 read 0xd4080ef thru 0xd40c0e0 (0x3ff2 bytes) 2024-09-18T01:10:20.539 INFO:teuthology.orchestra.run.smithi120.stdout:4954 read 0x21a2e66 thru 0x21b1a74 (0xec0f bytes) 2024-09-18T01:10:20.539 INFO:teuthology.orchestra.run.smithi120.stdout:4955 write 0x31d5d95 thru 0x31da5df (0x484b bytes) 2024-09-18T01:10:20.548 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.548 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:20.548 INFO:teuthology.orchestra.run.smithi120.stdout:4959 read 0x5935af4 thru 0x5942ecf (0xd3dc bytes) 2024-09-18T01:10:20.548 INFO:teuthology.orchestra.run.smithi120.stdout:4961 writesame 0x8e117c4 thru 0x8e1c300 data_size 0x1(0xab3d bytes) 2024-09-18T01:10:20.556 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:20.556 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:20.556 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:20.556 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:20.556 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:20.567 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:20.568 INFO:teuthology.orchestra.run.smithi120.stdout:4962 write 0x86eec3 thru 0x870be5 (0x1d23 bytes) 2024-09-18T01:10:20.575 INFO:teuthology.orchestra.run.smithi120.stdout:4963 read 0x80a8c3d thru 0x80aa83b (0x1bff bytes) 2024-09-18T01:10:20.575 INFO:teuthology.orchestra.run.smithi120.stdout:4964 write 0xae91873 thru 0xae919e3 (0x171 bytes) 2024-09-18T01:10:20.579 INFO:teuthology.orchestra.run.smithi120.stdout:4965 read 0xbdea60c thru 0xbdf86a2 (0xe097 bytes) 2024-09-18T01:10:20.579 INFO:teuthology.orchestra.run.smithi120.stdout:4966 punch from 0x934e04d to 0x934e242, (0x1f5 bytes) 2024-09-18T01:10:20.586 INFO:teuthology.orchestra.run.smithi120.stderr:marked out osd.2. 2024-09-18T01:10:20.593 INFO:teuthology.orchestra.run.smithi120.stdout:4968 trunc from 0xdc0fc86 to 0x288b166 2024-09-18T01:10:20.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:10:20.606 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:20.605+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:20.619 INFO:teuthology.orchestra.run.smithi120.stdout:4969 write 0xdc0e56 thru 0xdc5788 (0x4933 bytes) 2024-09-18T01:10:20.621 INFO:teuthology.orchestra.run.smithi120.stdout:4972 punch from 0x1f4d76d to 0x1f5ab24, (0xd3b7 bytes) 2024-09-18T01:10:20.639 INFO:teuthology.orchestra.run.smithi120.stdout:4973 write 0xc08f665 thru 0xc0949f6 (0x5392 bytes) 2024-09-18T01:10:20.681 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:20.681 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.95678702999999, 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.682 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:20.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:20.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925760508, 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:20.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.92575093, 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:20.695 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:20.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:20.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925736904000001, 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:20.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:20.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925708131, 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:20.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.713 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925680435, 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:20.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.719 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925645986999999, 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:20.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:20.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925616140999999, 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.92559395, 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925557044000001, 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925535928999999, 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.925505403999999, 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.920025150000001, 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919971291, 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919893868999999, 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919863755000002, 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:20.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919813837, 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919780249999999, 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919755716000001, 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.919570434000001, 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:20.807 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:20.808 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:20.809 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:20.807+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:20.909 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:20.908+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:20.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:20.912 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:20.920 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:21.010 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:21.009+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:21.065 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.066 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:21.066 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:21.066 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:21.066 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:21.118 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:21.117+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:21.178 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.577442634000001, 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:21.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576292619, 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:21.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576273308000001, 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576245562, 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.195 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:21.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:21.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:21.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:21.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:21.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:21.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576225709999999, 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:21.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:21.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:21.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576205846000001, 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:21.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576187640000001, 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576167593000001, 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576149385000001, 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576131273000001, 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576113355, 2024-09-18T01:10:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.576095335000002, 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575542379000002, 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575523656000001, 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575493872999999, 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575429785000001, 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575245484, 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575227330000001, 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575198958000001, 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 22.575177031999999, 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.313 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:21.315 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:21.314+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:21.416 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:21.415+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:21.459 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.460 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:21.460 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:21.460 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.468 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:21.517 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:21.516+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:21.572 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:21.602 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:10:21.618 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:21.616+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:21.719 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:21.718+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:21.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.15293129, 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:21.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:21.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:21.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.100940147, 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:21.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.100863351, 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:21.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.096358198, 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:21.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:21.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:21.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.096102402, 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:21.752 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:21.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:21.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:21.757 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.092077074, 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:21.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.091935924, 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:21.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.09170302, 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:21.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:21.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.775 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.089083958, 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:21.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:21.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:21.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:21.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:21.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.06674492400001, 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:21.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:21.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:21.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116229649999994, 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:21.796 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.797 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:21.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.798 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116207908999996, 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116187500999999, 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116167841999996, 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116134506999998, 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116099974999997, 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116055084999999, 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:21.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:21.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.848 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116028709999995, 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:21.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.116009559000005, 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:21.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.115990991000004, 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:21.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:21.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:21.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:21.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:21.864 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:21.865 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:21.865+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:21.967 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:21.966+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:21.967 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:21.968 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:21.978 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:21.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:22.068 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:22.067+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:22.080 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:22.080 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:22.080 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:22.081 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:22.081 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:22.089 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:22.115 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:22.132 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:22.148 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:22.148 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:22.164 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:22.164 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:22.173 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:22.172+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:22.180 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:22.180 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:22.196 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:22.196 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:22.212 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:22.213 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:22.229 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:22.233 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.234 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:22.283 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:22.282+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:22.344 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.345 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:22.397 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:22.396+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:22.455 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.455 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.463 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:22.505 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:22.504+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:22.572 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:22.571+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:22.613 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.613 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:22.672 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:22.671+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:22.724 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.724 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:22.778 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:22.772+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:22.845 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.845 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.846 INFO:teuthology.orchestra.run.smithi120.stdout:4974 punch from 0x17c4328 to 0x17ce0e2, (0x9dba bytes) 2024-09-18T01:10:22.846 INFO:teuthology.orchestra.run.smithi120.stdout:4975 write 0x306d201 thru 0x306e8bf (0x16bf bytes) 2024-09-18T01:10:22.846 INFO:teuthology.orchestra.run.smithi120.stdout:4976 punch from 0x5dd75cc to 0x5ddd1bd, (0x5bf1 bytes) 2024-09-18T01:10:22.846 INFO:teuthology.orchestra.run.smithi120.stdout:4978 writesame 0x2ea11f thru 0x2f5fbe data_size 0x3d(0xbea0 bytes) 2024-09-18T01:10:22.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:22.857 INFO:teuthology.orchestra.run.smithi120.stdout:4979 write 0x1f4c04a thru 0x1f59e87 (0xde3e bytes) 2024-09-18T01:10:22.873 INFO:teuthology.orchestra.run.smithi120.stdout:4980 trunc from 0xc0949f7 to 0x3818c8c 2024-09-18T01:10:22.874 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:22.873+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:22.894 INFO:teuthology.orchestra.run.smithi120.stdout:4983 punch from 0x495af9 to 0x4a16e0, (0xbbe7 bytes) 2024-09-18T01:10:22.907 INFO:teuthology.orchestra.run.smithi120.stdout:4986 write 0x890f051 thru 0x89135fb (0x45ab bytes) 2024-09-18T01:10:22.922 INFO:teuthology.orchestra.run.smithi120.stdout:4987 write 0x44412b7 thru 0x4441492 (0x1dc bytes) 2024-09-18T01:10:22.927 INFO:teuthology.orchestra.run.smithi120.stdout:4988 write 0x4186fb1 thru 0x418b5ca (0x461a bytes) 2024-09-18T01:10:22.948 INFO:teuthology.orchestra.run.smithi120.stdout:4990 write 0xe96aae1 thru 0xe9765f7 (0xbb17 bytes) 2024-09-18T01:10:22.964 INFO:teuthology.orchestra.run.smithi120.stdout:4991 trunc from 0xe9765f8 to 0xb84ce1a 2024-09-18T01:10:22.969 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:22.969 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:22.974 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:22.973+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:22.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:22.987 INFO:teuthology.orchestra.run.smithi120.stdout:4993 trunc from 0xb84ce1a to 0x98fb7fa 2024-09-18T01:10:22.993 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:22.993 INFO:teuthology.orchestra.run.smithi120.stdout:4996 read 0x8517041 thru 0x851931d (0x22dd bytes) 2024-09-18T01:10:22.993 INFO:teuthology.orchestra.run.smithi120.stdout:4997 read 0x30ed536 thru 0x30f1ccd (0x4798 bytes) 2024-09-18T01:10:22.993 INFO:teuthology.orchestra.run.smithi120.stdout:4999 write 0x350c741 thru 0x35193cb (0xcc8b bytes) 2024-09-18T01:10:23.005 INFO:teuthology.orchestra.run.smithi120.stdout:5002 write 0x5cd73cf thru 0x5cda839 (0x346b bytes) 2024-09-18T01:10:23.024 INFO:teuthology.orchestra.run.smithi120.stdout:5004 writesame 0xeb517fc thru 0xeb5feb6 data_size 0x3(0xe6bb bytes) 2024-09-18T01:10:23.024 INFO:teuthology.orchestra.run.smithi120.stdout:5005 read 0x12f74f5 thru 0x12fe4be (0x6fca bytes) 2024-09-18T01:10:23.025 INFO:teuthology.orchestra.run.smithi120.stdout:5006 writesame 0x901893f thru 0x9028320 data_size 0x5(0xf9e2 bytes) 2024-09-18T01:10:23.038 INFO:teuthology.orchestra.run.smithi120.stdout:5007 writesame 0xfd33cb thru 0xfd4c2b data_size 0x4f(0x1861 bytes) 2024-09-18T01:10:23.044 INFO:teuthology.orchestra.run.smithi120.stdout:5008 clone 40 order 21 su 524288 sc 3 2024-09-18T01:10:23.075 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:23.074+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:23.099 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.099 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.108 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:23.175 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:23.174+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:23.260 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.260 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:23.276 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:23.275+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:23.377 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:23.376+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:23.377 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.377 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:23.477 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:23.476+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:23.530 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.530 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.539 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:23.583 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:23.582+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:23.618 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:10:23.649 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.650 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.660 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:23.680 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:23.678+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:23.780 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:23.779+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:23.805 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.805 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.815 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:23.881 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:23.880+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:23.918 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:23.918 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:23.927 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:23.982 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:23.981+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:24.029 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:24.029 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:24.038 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:24.083 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:24.082+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:24.180 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:24.180 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:24.184 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:24.183+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:24.188 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:24.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:24.285 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:24.284+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:24.331 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:24.332 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:24.340 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:24.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:24.379 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:24.394 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:24.395 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:24.410 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:24.410 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:24.424 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:24.423+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:24.426 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:24.443 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:24.443 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:24.459 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:24.459 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:24.475 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:24.476 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:24.485 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:24.485 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:24.486 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:24.486+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:24.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:24.570 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:10:24.588 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:24.587+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:24.673 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.071985414, 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070835399, 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070816088000001, 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070788342, 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070768489999999, 2024-09-18T01:10:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070748626, 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.07073042, 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070710373000001, 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:24.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070692165000001, 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:24.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070674053000001, 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.717 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:24.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070656135, 2024-09-18T01:10:24.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:24.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070638115000001, 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:24.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:24.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070085159000001, 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:24.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:24.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070066436000001, 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.739 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.070036652999999, 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:24.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.069972565, 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:24.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.069788264, 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:24.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.06977011, 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:24.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:24.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.069741738000001, 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:24.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.069719811999999, 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:24.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:24.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:24.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:24.767 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:24.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:24.769 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:24.767+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:24.869 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:24.868+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:24.914 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:24.914 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:24.922 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:25.006 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:25.005+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:25.066 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:25.066 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:25.071 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:25.070+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:25.075 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:25.172 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:25.171+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.219 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.646460717, 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:25.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:25.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.594469574, 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:25.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.594392778, 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:25.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.589887625, 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:25.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:25.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.589631829, 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:25.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.585606501, 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:25.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:25.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.585465351, 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.253 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.585232447, 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.258 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:25.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:25.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.582613385, 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:25.263 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:25.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:25.268 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.560274351, 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:25.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609759077000007, 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:25.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609737335999995, 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:25.278 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:25.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:25.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:25.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609716927999997, 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:25.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:25.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:25.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:25.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609697268999994, 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:25.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609663933999997, 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:25.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:25.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:25.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609629401999996, 2024-09-18T01:10:25.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:25.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:25.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609584511999998, 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.303 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609558136999993, 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:25.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609538986000004, 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:25.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.609520418000002, 2024-09-18T01:10:25.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:25.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:25.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.325 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:25.327 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:25.327 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:25.326+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:25.428 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:25.427+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:25.430 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:25.430 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:25.439 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:25.529 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:25.528+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:25.544 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:25.544 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:25.544 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:25.544 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:25.544 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:25.553 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:25.607 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:25.607 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:25.607 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T01:10:25.607 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:10:25.607 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:10:25.608 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T01:10:25.652 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:25.651+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:25.703 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 193.80390671999999, 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:25.705 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:25.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:25.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:25.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:25.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:25.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.079978350000005, 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.715 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:25.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 92.079956168999999, 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:25.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:25.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.112451702000001, 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:25.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.731 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.107479373, 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:25.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:25.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.736 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.102586989999999, 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:25.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:25.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.741 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.097172352000001, 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:25.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.085598086000001, 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:25.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:25.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.076630392999999, 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.752 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:25.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.076575931000001, 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.757 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:25.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.071304557000001, 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:25.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:25.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.071223818, 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:25.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:25.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.070262819, 2024-09-18T01:10:25.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:25.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:25.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.070204026999999, 2024-09-18T01:10:25.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:25.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.069040058999999, 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:25.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:25.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.068675389999999, 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:25.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:25.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:25.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.068152854000001, 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:25.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:25.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.067755472999998, 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:25.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:25.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.065902742999999, 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:25.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:25.809 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.065780552, 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:25.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:25.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:25.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:25.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:25.815 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:25.816 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:25.817 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:10:25.817 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:25.817 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-18T01:10:25.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:25.818 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:25.815+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:25.917 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:25.916+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:25.964 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:25.964 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:25.964 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:25.964 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:25.973 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:25.991 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:25.991 INFO:teuthology.orchestra.run.smithi120.stdout:5010 write 0xab940fc thru 0xab956cf (0x15d4 bytes) 2024-09-18T01:10:26.000 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:26.000 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:26.001 INFO:teuthology.orchestra.run.smithi120.stdout:5013 read 0x55a8b01 thru 0x55aaa02 (0x1f02 bytes) 2024-09-18T01:10:26.001 INFO:teuthology.orchestra.run.smithi120.stdout:5014 writesame 0x67eaae4 thru 0x67f6224 data_size 0x1(0xb741 bytes) 2024-09-18T01:10:26.017 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:26.016+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:26.024 INFO:teuthology.orchestra.run.smithi120.stdout:5015 read 0x35edb06 thru 0x35f45f4 (0x6aef bytes) 2024-09-18T01:10:26.024 INFO:teuthology.orchestra.run.smithi120.stdout:5016 write 0x64370f9 thru 0x64428ee (0xb7f6 bytes) 2024-09-18T01:10:26.028 INFO:teuthology.orchestra.run.smithi120.stdout:5017 punch from 0x1e8cbc2 to 0x1e903e1, (0x381f bytes) 2024-09-18T01:10:26.045 INFO:teuthology.orchestra.run.smithi120.stdout:5019 trunc from 0xeb5feb7 to 0x5b2b5f 2024-09-18T01:10:26.070 INFO:teuthology.orchestra.run.smithi120.stdout:5020 read 0xaf31e thru 0xb6d7b (0x7a5e bytes) 2024-09-18T01:10:26.075 INFO:teuthology.orchestra.run.smithi120.stdout:5021 trunc from 0x5b2b5f to 0x3b6e6c3 2024-09-18T01:10:26.090 INFO:teuthology.orchestra.run.smithi120.stdout:5022 write 0x65953bd thru 0x65a2bcf (0xd813 bytes) 2024-09-18T01:10:26.093 INFO:teuthology.orchestra.run.smithi120.stdout:5025 read 0x116f004 thru 0x117cd4c (0xdd49 bytes) 2024-09-18T01:10:26.098 INFO:teuthology.orchestra.run.smithi120.stdout:5026 trunc from 0x65a2bd0 to 0x9f138d5 2024-09-18T01:10:26.103 INFO:teuthology.orchestra.run.smithi120.stdout:5027 read 0x4585366 thru 0x4594b13 (0xf7ae bytes) 2024-09-18T01:10:26.104 INFO:teuthology.orchestra.run.smithi120.stdout:5028 punch from 0x26f0db0 to 0x26fd8f5, (0xcb45 bytes) 2024-09-18T01:10:26.108 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:26.109 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:26.109 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:26.109 INFO:teuthology.orchestra.run.smithi120.stdout:5029 write 0xa384adb thru 0xa394a82 (0xffa8 bytes) 2024-09-18T01:10:26.114 INFO:teuthology.orchestra.run.smithi120.stdout:5031 write 0x54a027d thru 0x54a2e86 (0x2c0a bytes) 2024-09-18T01:10:26.123 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:26.123 INFO:teuthology.orchestra.run.smithi120.stdout:5032 read 0x7848023 thru 0x7848e70 (0xe4e bytes) 2024-09-18T01:10:26.123 INFO:teuthology.orchestra.run.smithi120.stdout:5033 read 0x65b925c thru 0x65bf1a2 (0x5f47 bytes) 2024-09-18T01:10:26.123 INFO:teuthology.orchestra.run.smithi120.stdout:5034 clone 41 order 19 su 262144 sc 13 2024-09-18T01:10:26.124 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:26.122+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:26.224 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:26.223+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:26.267 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:26.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 148.54260128199999, 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.268 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:26.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:26.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.519143192000001, 2024-09-18T01:10:26.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:26.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.506064940000002, 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.506044682999999, 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.285 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:26.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505699648, 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:26.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505392518000001, 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:26.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:26.300 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505356759000001, 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:26.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:26.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:26.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.305 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505318490000001, 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:26.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:26.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505294018000001, 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.311 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:26.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.315 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505273357, 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.316 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:26.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505254733000001, 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.321 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:26.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:26.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505234942000001, 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:26.326 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:26.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505215839000002, 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505197749000001, 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505179649999999, 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505161320999999, 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505142955, 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505124694999999, 2024-09-18T01:10:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505106241, 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.505056554999999, 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:26.376 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:26.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:26.378 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:26.376+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:26.477 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:26.476+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:26.487 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:26.523 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:26.523 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:26.523 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:26.523 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:26.532 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:26.579 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:26.578+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:26.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:10:26.612 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:26.612 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:26.633 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:26.633 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:26.652 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:26.652 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:26.670 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:26.679 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:26.678+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:26.687 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:26.687 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:26.687 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:26.687 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:26.697 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:26.705 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:26.705 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:26.722 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:26.722 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:26.740 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:26.741 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:26.780 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:26.779+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:26.831 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:26.831 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:26.840 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:26.885 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:26.884+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:26.946 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:26.946 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:26.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:26.996 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:26.995+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:27.084 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:27.083+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:27.097 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.098 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:27.185 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:27.184+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:27.208 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.208 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.217 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:27.301 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:27.300+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:27.360 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.360 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:27.410 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:27.409+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:27.487 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:27.487+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:27.511 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.511 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:27.589 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:27.588+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:27.628 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:10:27.680 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.680 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.690 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:27.690 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:27.688+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:27.790 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:27.789+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:27.799 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.799 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.808 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:27.892 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:27.891+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:27.951 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:27.951 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:27.959 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:27.993 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:27.992+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:28.093 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:28.093+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:28.101 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.101 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.110 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:28.194 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:28.193+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:28.213 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.213 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:28.306 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:28.305+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:28.365 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.365 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:28.397 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:28.396+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:28.498 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:28.497+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:28.516 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.516 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:28.598 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:28.598+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:28.628 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.628 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:28.700 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:28.699+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:28.743 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.743 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:28.752 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:28.768 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-18T01:10:28.801 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:28.800+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:28.860 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.861 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:28.900 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:28.900 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:28.901 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:28.900+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:28.917 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:28.917 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:28.934 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:28.934 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:28.950 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:28.951 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:28.967 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:28.968 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:28.974 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:28.974 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:28.983 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:28.985 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:28.985 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:29.001 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:29.002 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:29.001+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:29.018 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:29.019 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:29.046 INFO:teuthology.orchestra.run.smithi120.stdout:5035 read 0x64fcd8f thru 0x650866e (0xb8e0 bytes) 2024-09-18T01:10:29.046 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.059 INFO:teuthology.orchestra.run.smithi120.stdout:5037 write 0xbad3bfa thru 0xbada010 (0x6417 bytes) 2024-09-18T01:10:29.063 INFO:teuthology.orchestra.run.smithi120.stdout:5038 writesame 0x3dfe652 thru 0x3e0189c data_size 0x19(0x324b bytes) 2024-09-18T01:10:29.079 INFO:teuthology.orchestra.run.smithi120.stdout:5039 writesame 0xeb05638 thru 0xeb089e3 data_size 0x1(0x33ac bytes) 2024-09-18T01:10:29.089 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:29.090 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:29.093 INFO:teuthology.orchestra.run.smithi120.stdout:5041 trunc from 0xeb089e4 to 0x2ce337e 2024-09-18T01:10:29.099 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:29.129 INFO:teuthology.orchestra.run.smithi120.stdout:5042 write 0xd6a7c71 thru 0xd6aaa0d (0x2d9d bytes) 2024-09-18T01:10:29.133 INFO:teuthology.orchestra.run.smithi120.stdout:5043 read 0x1d61497 thru 0x1d680b6 (0x6c20 bytes) 2024-09-18T01:10:29.144 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:29.143+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:29.148 INFO:teuthology.orchestra.run.smithi120.stdout:5045 trunc from 0xd6aaa0e to 0x19b137 2024-09-18T01:10:29.192 INFO:teuthology.orchestra.run.smithi120.stdout:5046 write 0xa460677 thru 0xa466275 (0x5bff bytes) 2024-09-18T01:10:29.196 INFO:teuthology.orchestra.run.smithi120.stdout:5047 write 0x354cae7 thru 0x35533d8 (0x68f2 bytes) 2024-09-18T01:10:29.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.200 INFO:teuthology.orchestra.run.smithi120.stdout:5052 writesame 0x7f53075 thru 0x7f603c1 data_size 0x49(0xd34d bytes) 2024-09-18T01:10:29.203 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:29.202+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:29.204 INFO:teuthology.orchestra.run.smithi120.stdout:5054 read 0x2f977a3 thru 0x2fa3427 (0xbc85 bytes) 2024-09-18T01:10:29.204 INFO:teuthology.orchestra.run.smithi120.stdout:5055 read 0x7bfc290 thru 0x7c00812 (0x4583 bytes) 2024-09-18T01:10:29.205 INFO:teuthology.orchestra.run.smithi120.stdout:5057 read 0x7cfa13 thru 0x7da45b (0xaa49 bytes) 2024-09-18T01:10:29.205 INFO:teuthology.orchestra.run.smithi120.stdout:5060 read 0x9e47f65 thru 0x9e4b39a (0x3436 bytes) 2024-09-18T01:10:29.205 INFO:teuthology.orchestra.run.smithi120.stdout:5061 read 0x5a94a91 thru 0x5a9e089 (0x95f9 bytes) 2024-09-18T01:10:29.206 INFO:teuthology.orchestra.run.smithi120.stdout:5063 trunc from 0xa466276 to 0x98a5891 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 197.31305956099999, 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.589131191000007, 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.589109010000001, 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.621604542999997, 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.616632213999999, 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.611739831000001, 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.606325193, 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.594750927, 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.585783234000001, 2024-09-18T01:10:29.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:29.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:29.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:29.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.585728772, 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:29.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:29.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.580457398, 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:29.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:29.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:29.272 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.580376658999999, 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:29.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.579415659999999, 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:29.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:29.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.579356868000001, 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:29.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.578192900000001, 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:29.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.577828231000002, 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.293 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:29.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.298 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.577305695, 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:29.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.576908314000001, 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.575055584000001, 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.574933392999998, 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:29.319 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:29.321 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:29.319+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:29.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:29.322 INFO:teuthology.orchestra.run.smithi120.stdout:5064 read 0x46302d0 thru 0x463f3e0 (0xf111 bytes) 2024-09-18T01:10:29.322 INFO:teuthology.orchestra.run.smithi120.stdout:5065 write 0x9f9c5f6 thru 0x9fa8f79 (0xc984 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5066 write 0xf806a5 thru 0xf85b6b (0x54c7 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5067 read 0x50b773 thru 0x519c64 (0xe4f2 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5068 read 0x9764b4c thru 0x977351a (0xe9cf bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5069 writesame 0x41b296a thru 0x41b566d data_size 0x2b(0x2d04 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5070 trunc from 0x9fa8f7a to 0x467a683 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5072 write 0x78d181f thru 0x78d7732 (0x5f14 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5073 trunc from 0x78d7733 to 0x56625e6 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5074 read 0x8fd64f thru 0x90a03c (0xc9ee bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5075 read 0x48fa91c thru 0x4901862 (0x6f47 bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5076 read 0xaba572 thru 0xac199d (0x742c bytes) 2024-09-18T01:10:29.323 INFO:teuthology.orchestra.run.smithi120.stdout:5077 writesame 0x46bd689 thru 0x46ccb46 data_size 0x1(0xf4be bytes) 2024-09-18T01:10:29.324 INFO:teuthology.orchestra.run.smithi120.stdout:5078 write 0xd6b8e1c thru 0xd6c43b3 (0xb598 bytes) 2024-09-18T01:10:29.328 INFO:teuthology.orchestra.run.smithi120.stdout:5079 punch from 0x58c9df4 to 0x58d818b, (0xe397 bytes) 2024-09-18T01:10:29.328 INFO:teuthology.orchestra.run.smithi120.stdout:5080 read 0x7b87bdb thru 0x7b8b9dc (0x3e02 bytes) 2024-09-18T01:10:29.329 INFO:teuthology.orchestra.run.smithi120.stdout:5081 read 0x727d027 thru 0x7286e70 (0x9e4a bytes) 2024-09-18T01:10:29.337 INFO:teuthology.orchestra.run.smithi120.stdout:5082 trunc from 0xd6c43b4 to 0x5a9700a 2024-09-18T01:10:29.355 INFO:teuthology.orchestra.run.smithi120.stdout:5083 write 0x713aef6 thru 0x71420ce (0x71d9 bytes) 2024-09-18T01:10:29.359 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.367 INFO:teuthology.orchestra.run.smithi120.stdout:5085 write 0x95deba2 thru 0x95ead39 (0xc198 bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5087 read 0x7c5500e thru 0x7c60af4 (0xbae7 bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5088 read 0x7000d9f thru 0x7005446 (0x46a8 bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5090 read 0x8d4d427 thru 0x8d4e2cc (0xea6 bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5091 read 0x13382d9 thru 0x133b904 (0x362c bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5093 read 0x680b5bc thru 0x680f8dd (0x4322 bytes) 2024-09-18T01:10:29.371 INFO:teuthology.orchestra.run.smithi120.stdout:5094 read 0x2e1b5c5 thru 0x2e24c3c (0x9678 bytes) 2024-09-18T01:10:29.383 INFO:teuthology.orchestra.run.smithi120.stdout:5095 write 0xeb99875 thru 0xeba3a07 (0xa193 bytes) 2024-09-18T01:10:29.399 INFO:teuthology.orchestra.run.smithi120.stdout:5096 trunc from 0xeba3a08 to 0x41d9ef0 2024-09-18T01:10:29.421 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:29.420+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:29.429 INFO:teuthology.orchestra.run.smithi120.stdout:5098 write 0x9775e19 thru 0x9783acb (0xdcb3 bytes) 2024-09-18T01:10:29.436 INFO:teuthology.orchestra.run.smithi120.stdout:5099 punch from 0xe76c11 to 0xe79039, (0x2428 bytes) 2024-09-18T01:10:29.437 INFO:teuthology.orchestra.run.smithi120.stdout:5100 read 0x50ea659 thru 0x50eb1e0 (0xb88 bytes) 2024-09-18T01:10:29.437 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.437 INFO:teuthology.orchestra.run.smithi120.stdout:5103 writesame 0x20a2a45 thru 0x20a3ac2 data_size 0x1(0x107e bytes) 2024-09-18T01:10:29.440 INFO:teuthology.orchestra.run.smithi120.stdout:5104 write 0x23b5fa2 thru 0x23bfe88 (0x9ee7 bytes) 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:241439 3.17 3:e8d2c611:::rbd_object_map.117653c0e20f:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:29.440811+0000", 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0010283880000000001, 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001044419, 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 241439 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440811+0000", 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440811+0000", 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440812+0000", 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4099999999999997e-07 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440817+0000", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6779999999999999e-06 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440826+0000", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4579999999999999e-06 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440831+0000", 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440869+0000", 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7215999999999997e-05 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.440922+0000", 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3729e-05 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-18T01:10:29.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:29.441138+0000", 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215553 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-09-18T01:10:29.447 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:5105 read 0x3b96567 thru 0x3ba0d67 (0xa801 bytes) 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:5112 read 0x41e2bb0 thru 0x41ee59b (0xb9ec bytes) 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.448 INFO:teuthology.orchestra.run.smithi120.stdout:5114 writesame 0x57b1b4e thru 0x57b97f6 data_size 0x2f(0x7ca9 bytes) 2024-09-18T01:10:29.452 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:29.453 INFO:teuthology.orchestra.run.smithi120.stdout:5115 write 0x9cd67ea thru 0x9cdef5d (0x8774 bytes) 2024-09-18T01:10:29.457 INFO:teuthology.orchestra.run.smithi120.stdout:5116 read 0x693862a thru 0x693ad74 (0x274b bytes) 2024-09-18T01:10:29.457 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.466 INFO:teuthology.orchestra.run.smithi120.stdout:5118 writesame 0xc37198b thru 0xc37b25c data_size 0x2(0x98d2 bytes) 2024-09-18T01:10:29.466 INFO:teuthology.orchestra.run.smithi120.stdout:5119 read 0x180b458 thru 0x180b8a0 (0x449 bytes) 2024-09-18T01:10:29.467 INFO:teuthology.orchestra.run.smithi120.stdout:5120 write 0x2b6b91c thru 0x2b790c6 (0xd7ab bytes) 2024-09-18T01:10:29.470 INFO:teuthology.orchestra.run.smithi120.stdout:5123 read 0x8a6e3ee thru 0x8a797b7 (0xb3ca bytes) 2024-09-18T01:10:29.470 INFO:teuthology.orchestra.run.smithi120.stdout:5124 write 0x1b2be1 thru 0x1b9b43 (0x6f63 bytes) 2024-09-18T01:10:29.478 INFO:teuthology.orchestra.run.smithi120.stdout:5125 trunc from 0xc37b25d to 0x6715463 2024-09-18T01:10:29.499 INFO:teuthology.orchestra.run.smithi120.stdout:5127 trunc from 0x6715463 to 0x6c7f14 2024-09-18T01:10:29.522 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:29.521+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:29.536 INFO:teuthology.orchestra.run.smithi120.stdout:5128 write 0xbdf4b68 thru 0xbdfe50e (0x99a7 bytes) 2024-09-18T01:10:29.544 INFO:teuthology.orchestra.run.smithi120.stdout:5129 trunc from 0xbdfe50f to 0x900a6cf 2024-09-18T01:10:29.555 INFO:teuthology.orchestra.run.smithi120.stdout:5131 read 0x6845721 thru 0x6854709 (0xefe9 bytes) 2024-09-18T01:10:29.556 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.556 INFO:teuthology.orchestra.run.smithi120.stdout:5133 read 0x245a736 thru 0x2461031 (0x68fc bytes) 2024-09-18T01:10:29.564 INFO:teuthology.orchestra.run.smithi120.stdout:5134 write 0xaa12f55 thru 0xaa17626 (0x46d2 bytes) 2024-09-18T01:10:29.569 INFO:teuthology.orchestra.run.smithi120.stdout:5135 read 0x99f4103 thru 0x99f4c86 (0xb84 bytes) 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout:5136 write 0x76f2657 thru 0x76f61fd (0x3ba7 bytes) 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:29.570 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:29.575 INFO:teuthology.orchestra.run.smithi120.stdout:5137 punch from 0x56a03b1 to 0x56a2f10, (0x2b5f bytes) 2024-09-18T01:10:29.575 INFO:teuthology.orchestra.run.smithi120.stdout:5138 read 0x7fdba92 thru 0x7fe2f70 (0x74df bytes) 2024-09-18T01:10:29.576 INFO:teuthology.orchestra.run.smithi120.stdout:5139 read 0x2363a64 thru 0x2364ad5 (0x1072 bytes) 2024-09-18T01:10:29.576 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.576 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.576 INFO:teuthology.orchestra.run.smithi120.stdout:5144 writesame 0x1aad480 thru 0x1ab3d02 data_size 0x1(0x6883 bytes) 2024-09-18T01:10:29.580 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:29.592 INFO:teuthology.orchestra.run.smithi120.stdout:5146 read 0x741d50f thru 0x74232e8 (0x5dda bytes) 2024-09-18T01:10:29.600 INFO:teuthology.orchestra.run.smithi120.stdout:5147 write 0xc243cc8 thru 0xc243fb0 (0x2e9 bytes) 2024-09-18T01:10:29.604 INFO:teuthology.orchestra.run.smithi120.stdout:5148 read 0x2f27a73 thru 0x2f34977 (0xcf05 bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5149 punch from 0x1ed64a to 0x1f579f, (0x8155 bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5151 read 0x28cd02b thru 0x28cffc3 (0x2f99 bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5153 read 0xb7194e6 thru 0xb724c71 (0xb78c bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5154 read 0x6e24a2b thru 0x6e33181 (0xe757 bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5155 read 0x8f5c6d6 thru 0x8f69772 (0xd09d bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5157 read 0x263bde3 thru 0x263edbc (0x2fda bytes) 2024-09-18T01:10:29.605 INFO:teuthology.orchestra.run.smithi120.stdout:5158 writesame 0x31d3786 thru 0x31d7698 data_size 0x1(0x3f13 bytes) 2024-09-18T01:10:29.622 INFO:teuthology.orchestra.run.smithi120.stdout:5159 trunc from 0xc243fb1 to 0x6024518 2024-09-18T01:10:29.623 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:29.622+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:29.643 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:10:29.648 INFO:teuthology.orchestra.run.smithi120.stdout:5160 writesame 0xded474c thru 0xded891e data_size 0x29(0x41d3 bytes) 2024-09-18T01:10:29.648 INFO:teuthology.orchestra.run.smithi120.stdout:5161 write 0x6564da3 thru 0x65667f2 (0x1a50 bytes) 2024-09-18T01:10:29.652 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.652 INFO:teuthology.orchestra.run.smithi120.stdout:5163 write 0xce8a487 thru 0xce93d9a (0x9914 bytes) 2024-09-18T01:10:29.655 INFO:teuthology.orchestra.run.smithi120.stdout:5165 punch from 0x81926ce to 0x819d78a, (0xb0bc bytes) 2024-09-18T01:10:29.656 INFO:teuthology.orchestra.run.smithi120.stdout:5166 trunc from 0xded891f to 0xd39c77d 2024-09-18T01:10:29.662 INFO:teuthology.orchestra.run.smithi120.stdout:5167 write 0x6ec23dc thru 0x6ec7bc7 (0x57ec bytes) 2024-09-18T01:10:29.670 INFO:teuthology.orchestra.run.smithi120.stdout:5168 read 0x2023b7 thru 0x20f7ba (0xd404 bytes) 2024-09-18T01:10:29.670 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.670 INFO:teuthology.orchestra.run.smithi120.stdout:5170 write 0x1e2c30a thru 0x1e2f46a (0x3161 bytes) 2024-09-18T01:10:29.673 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.673 INFO:teuthology.orchestra.run.smithi120.stdout:5172 write 0x103fa62 thru 0x1040b56 (0x10f5 bytes) 2024-09-18T01:10:29.678 INFO:teuthology.orchestra.run.smithi120.stdout:5173 punch from 0x86cbb7f to 0x86d9e08, (0xe289 bytes) 2024-09-18T01:10:29.678 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.678 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:29.679 INFO:teuthology.orchestra.run.smithi120.stdout:5176 writesame 0x639b8bf thru 0x639fe7c data_size 0x2(0x45be bytes) 2024-09-18T01:10:29.685 INFO:teuthology.orchestra.run.smithi120.stdout:5178 punch from 0x4581b94 to 0x4588a73, (0x6edf bytes) 2024-09-18T01:10:29.685 INFO:teuthology.orchestra.run.smithi120.stdout:5179 punch from 0x2770e09 to 0x2778f48, (0x813f bytes) 2024-09-18T01:10:29.685 INFO:teuthology.orchestra.run.smithi120.stdout:5180 clone 42 order 19 su 4096 sc 10 2024-09-18T01:10:29.708 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 151.983915981, 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:29.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:29.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:29.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.960457890999997, 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:29.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:29.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:29.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.947379638999998, 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:29.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:29.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.947359382000002, 2024-09-18T01:10:29.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:29.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:29.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:29.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.947014347, 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:29.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:29.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:29.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946707216999997, 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:29.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:29.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:29.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946671457999997, 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:29.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:29.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946633189000003, 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:29.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:29.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:29.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946608716999997, 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:29.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946588056000003, 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.765 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:29.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:29.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946569431999997, 2024-09-18T01:10:29.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:29.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:29.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:29.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946549640999997, 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:29.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:29.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946530537999998, 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:29.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:29.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946512448, 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:29.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:29.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:29.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.794 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946494348999998, 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:29.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946476019999999, 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:29.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:29.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946457654, 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:29.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:29.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:29.811 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946439394000002, 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:29.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946420940000003, 2024-09-18T01:10:29.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:29.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.946371253999999, 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:29.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:29.828 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:29.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:29.829 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:29.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:29.831 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:29.829+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:29.931 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:29.930+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:29.975 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:29.975 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:29.975 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:29.976 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:29.984 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:30.068 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:30.067+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:30.126 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:30.126 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:30.133 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:30.132+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:30.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:30.234 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:30.233+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:30.278 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:30.278 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.364602607999998, 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.279 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.364394576999999, 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:30.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:30.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350358002, 2024-09-18T01:10:30.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:30.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:30.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.292 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350334654000001, 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.293 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:30.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:30.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:30.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350313133, 2024-09-18T01:10:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:30.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.303 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350292413999998, 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:30.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:30.309 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350271637999999, 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:30.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:30.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.314 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350251493000002, 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:30.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350231117, 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.319 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:30.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350210389000001, 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:30.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:30.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.328 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350185930999999, 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:30.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:30.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.350165453999999, 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:30.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:30.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341389258, 2024-09-18T01:10:30.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:30.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:30.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341364257999999, 2024-09-18T01:10:30.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:30.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.349 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341327103000001, 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:30.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.34130042, 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:30.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:30.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341280521000002, 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:30.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341261314, 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341242301000001, 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.367 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.341224027999999, 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:30.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.377 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:30.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:30.379 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:30.377+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:30.479 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:30.478+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:30.522 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:30.522 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:30.522 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:30.522 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:30.531 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:30.579 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:30.578+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:30.634 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:30.634 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:30.634 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:30.634 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:30.634 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:30.639 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:10:30.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:30.680 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:30.679+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:30.781 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:30.780+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:30.798 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:30.798 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.135730778999999, 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.799 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.135676554, 2024-09-18T01:10:30.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:30.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:30.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.135644096, 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:30.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.135512833, 2024-09-18T01:10:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:30.816 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:30.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.135042847000001, 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:30.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.827 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:30.828 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.133880247, 2024-09-18T01:10:30.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:30.828 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.828 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.828 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.829 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:30.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:30.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:30.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.838 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.838 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:30.838 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.133772964999999, 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:30.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.841 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:30.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:30.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:30.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:30.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.849 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.133691306000003, 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:30.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:30.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:30.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.133175917999999, 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.855 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:30.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:30.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:30.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.132356680000001, 2024-09-18T01:10:30.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:30.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:30.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:30.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:30.869 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.559091594000002, 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:30.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:30.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:30.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:30.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:30.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:30.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.392232702000001, 2024-09-18T01:10:30.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.879 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:30.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:30.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:30.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.364387313999998, 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.887 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:30.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:30.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.364367014999999, 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.896 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:30.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:30.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:30.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.364346752000003, 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.906 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:30.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.364326898999998, 2024-09-18T01:10:30.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:30.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:30.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:30.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:30.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.364307742000001, 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:30.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:30.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.933 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 46.980317315999997, 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:30.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:30.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:30.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 46.920358452999999, 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:30.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:30.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:30.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:30.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:30.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.135855007, 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:30.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:30.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:30.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:30.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:30.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:30.957 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:30.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:30.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:30.958 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-18T01:10:30.958 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:10:30.958 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:10:30.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-18T01:10:30.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:30.961 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:30.959+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:31.030 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:31.061 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:31.060+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:31.074 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:31.075 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "" 2024-09-18T01:10:31.075 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:31.106 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:31.106 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:31.106 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:31.106 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:31.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:31.162 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:31.161+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:31.167 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:31.168 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:31.185 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:31.201 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:31.201 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:31.217 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:31.218 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:31.223 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.223 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.232 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:31.234 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:31.235 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:31.251 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:31.252 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:31.263 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:31.262+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:31.268 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:31.269 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:31.285 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:31.285 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:31.339 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.339 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.348 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:31.363 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:31.363+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:31.454 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.454 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:31.464 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:31.463+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:31.565 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:31.564+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:31.567 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.567 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:31.667 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:31.666+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:31.760 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.761 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.768 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:31.767+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:31.774 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:31.869 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:31.868+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:31.912 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:31.912 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:31.921 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:31.970 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:31.969+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:32.024 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.025 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:32.075 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:32.074+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:32.172 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:32.171+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:32.178 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.178 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:32.273 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:32.272+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:32.290 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.290 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.299 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:32.373 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:32.372+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:32.402 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.402 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.410 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:32.474 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:32.474+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:32.513 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.513 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.522 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:32.575 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:32.575+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:32.626 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.626 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:32.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:10:32.679 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:32.678+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:32.742 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.742 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:32.793 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:32.792+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:32.805 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-18T01:10:32.877 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:32.877+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:32.898 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:32.898 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:32.907 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:32.978 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:32.977+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:32.991 INFO:teuthology.orchestra.run.smithi120.stdout:5181 writesame 0x7917fa0 thru 0x7926a68 data_size 0x5(0xeac9 bytes) 2024-09-18T01:10:33.013 INFO:teuthology.orchestra.run.smithi120.stdout:5184 write 0x4ddc3bd thru 0x4deb8bc (0xf500 bytes) 2024-09-18T01:10:33.021 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:33.021 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:33.030 INFO:teuthology.orchestra.run.smithi120.stdout:5185 write 0x675e56 thru 0x68226f (0xc41a bytes) 2024-09-18T01:10:33.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:33.048 INFO:teuthology.orchestra.run.smithi120.stdout:5186 trunc from 0xd39c77d to 0xa25bd23 2024-09-18T01:10:33.069 INFO:teuthology.orchestra.run.smithi120.stdout:5187 read 0x9745477 thru 0x974ee29 (0x99b3 bytes) 2024-09-18T01:10:33.069 INFO:teuthology.orchestra.run.smithi120.stdout:5188 write 0x83bcbe8 thru 0x83cb8a0 (0xecb9 bytes) 2024-09-18T01:10:33.079 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:33.078+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:33.082 INFO:teuthology.orchestra.run.smithi120.stdout:5189 clone 43 order 25 su 16777216 sc 15 2024-09-18T01:10:33.155 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:33.155 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:33.163 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:33.180 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:33.179+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:33.281 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:33.280+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:33.297 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:33.307 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:33.307 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:33.317 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:33.381 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:33.380+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:33.419 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:33.419 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:33.436 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:33.436 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:33.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:33.452 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:33.464 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:33.464 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:33.464 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:33.464 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:33.468 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:33.468 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:33.474 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:33.486 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:33.486 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:33.485+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:33.503 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:33.503 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:33.519 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:33.536 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:33.578 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:33.579 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:33.583 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:33.582+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:33.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:33.653 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:10:33.684 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:33.683+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.825593667, 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:33.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.825385636, 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:33.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811349061000001, 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:33.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811325713000002, 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:33.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811304192000001, 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:33.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.762 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811283473000003, 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:33.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:33.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811262696999997, 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:33.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:33.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811242552000003, 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:33.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811222176000001, 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:33.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811201447999998, 2024-09-18T01:10:33.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:33.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.785 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.81117699, 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:33.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.811156513, 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.791 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802380317000001, 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:33.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802355317, 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:33.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:33.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:33.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802318161999999, 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.804 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:33.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802291478999997, 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.808 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.812 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802271580000003, 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:33.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802252373000002, 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802233360000002, 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.825 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.802215087, 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:33.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:33.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.830 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.830 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:33.830 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:33.830 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:33.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:33.832 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:33.830+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:33.932 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:33.931+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:33.935 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:33.935 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:33.935 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:33.935 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:33.943 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:34.033 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:34.032+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:34.046 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:34.046 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:34.046 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:34.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:34.134 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:34.133+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:34.158 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.496006717999997, 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.495952492999997, 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.495920034999997, 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.495788771999997, 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.495318785999999, 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.494156185999998, 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.494048904000003, 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.493967245, 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.493451856999997, 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.492632618999998, 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:34.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:34.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:34.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.919367532999999, 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.206 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:34.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.752508640999999, 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.212 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:34.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:34.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.724663253000003, 2024-09-18T01:10:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.724642953999997, 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:34.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:34.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.229 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.724622691, 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.724602838000003, 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.724583680999999, 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.241 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:34.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:34.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.340593255000002, 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.247 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:34.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:34.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:34.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:34.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.252 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.280634392000003, 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:34.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:34.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.496130946000001, 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.263 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:34.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:34.265 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:34.263+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:34.365 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:34.364+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:34.407 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:34.407 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:34.407 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:34.408 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:34.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:34.465 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:34.465+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:34.559 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:34.559 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:34.559 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:34.566 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:34.565+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:34.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:34.668 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:34.667+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:34.724 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062537074000002, 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:10:34.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:10:34.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062517878000001, 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:10:34.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062499516999999, 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:10:34.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:34.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062481413, 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:10:34.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062462816, 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:10:34.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:10:34.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062439816000001, 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:10:34.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062421144000002, 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062401140999999, 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:34.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:10:34.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062382900999999, 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:34.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:10:34.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:34.768 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.062364487, 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:10:34.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.06234246, 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.773 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:10:34.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058090680999999, 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:10:34.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:10:34.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:10:34.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058080229000002, 2024-09-18T01:10:34.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:10:34.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:10:34.786 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058068538000001, 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.787 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:10:34.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058055241000002, 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.792 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058039170000001, 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:10:34.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058022512000001, 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.801 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:10:34.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.058003217, 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:10:34.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.057974987000001, 2024-09-18T01:10:34.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:10:34.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.057953863000002, 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:34.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:34.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:10:34.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:10:34.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:34.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:34.820 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:34.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:34.822 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:34.820+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:34.922 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:34.922+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:34.932 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:34.932 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:34.932 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:34.932 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:34.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:35.023 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:35.022+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:35.045 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:35.045 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:35.045 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:35.045 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:35.045 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:35.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:35.123 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:35.123+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 157.434328475, 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403301953000003, 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403292374999999, 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403278348999997, 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403249576, 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403221879999997, 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403187432000003, 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403157585999999, 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403135395, 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403098489000001, 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403077373999999, 2024-09-18T01:10:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.403046848999999, 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397566595000001, 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397512736000003, 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397435313999999, 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397405200000001, 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:35.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397355281999999, 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:35.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:35.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397321695000002, 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:35.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397297160999997, 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:35.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:35.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.397111879000001, 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:35.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.269 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:35.271 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:35.271 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:35.270+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:35.372 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:35.371+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:35.415 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:35.415 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:35.415 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:35.416 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:35.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:35.508 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:35.507+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:35.548 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:35.567 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:35.567 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:35.567 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:35.567 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:35.567 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:35.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:35.593 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:35.592+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:35.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:10:35.675 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:35.674+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:35.687 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:35.705 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:35.706 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.122557260999997, 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:35.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121407245999997, 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:35.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121387935000001, 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:35.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:35.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.738 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121360189000001, 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:35.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:35.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121340336999999, 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:35.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121320472999997, 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.748 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:35.752 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121302266999997, 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.753 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:35.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121282219999998, 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121264011999997, 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:35.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:35.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121245899999998, 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121227982000001, 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:35.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.121209962000002, 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:35.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:35.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120657006000002, 2024-09-18T01:10:35.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:35.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120638282999998, 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:35.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:35.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120608500000003, 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:35.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:35.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120544412000001, 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:35.794 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:35.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120360110999997, 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:35.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:35.803 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120341957000001, 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120313584999998, 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.808 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:35.812 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.120291659000003, 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:35.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:35.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:35.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:35.817 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:35.818 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:35.818 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:35.819 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:35.819 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:35.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:35.820 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:35.817+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:35.820 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:35.820 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:35.821 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:35.821 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:35.821 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-18T01:10:35.821 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:35.822 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:35.822 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:35.822 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:35.919 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:35.918+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:35.963 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:35.963 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:35.972 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:36.019 INFO:teuthology.orchestra.run.smithi120.stdout:5190 read 0x534d2bc thru 0x535290a (0x564f bytes) 2024-09-18T01:10:36.020 INFO:teuthology.orchestra.run.smithi120.stdout:5191 read 0x9f467ec thru 0x9f50bcc (0xa3e1 bytes) 2024-09-18T01:10:36.020 INFO:teuthology.orchestra.run.smithi120.stdout:5192 read 0x5c43ae0 thru 0x5c4a21c (0x673d bytes) 2024-09-18T01:10:36.029 INFO:teuthology.orchestra.run.smithi120.stdout:5193 trunc from 0xa25bd23 to 0x32cf138 2024-09-18T01:10:36.054 INFO:teuthology.orchestra.run.smithi120.stdout:5194 trunc from 0x32cf138 to 0x74feabc 2024-09-18T01:10:36.057 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:36.056+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:36.067 INFO:teuthology.orchestra.run.smithi120.stdout:5195 trunc from 0x74feabc to 0x211418b 2024-09-18T01:10:36.085 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:36.085 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:36.085 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:10:36.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:10:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:5196 write 0xbc6f397 thru 0xbc6fd4c (0x9b6 bytes) 2024-09-18T01:10:36.110 INFO:teuthology.orchestra.run.smithi120.stdout:5197 trunc from 0xbc6fd4d to 0x51a0ab7 2024-09-18T01:10:36.117 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.118 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.120 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:36.119+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:36.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:36.137 INFO:teuthology.orchestra.run.smithi120.stdout:5199 writesame 0xbd13fa6 thru 0xbd172f3 data_size 0x21(0x334e bytes) 2024-09-18T01:10:36.137 INFO:teuthology.orchestra.run.smithi120.stdout:5200 write 0x278412a thru 0x2789f0c (0x5de3 bytes) 2024-09-18T01:10:36.139 INFO:teuthology.orchestra.run.smithi120.stdout:5203 write 0x6947e09 thru 0x6952404 (0xa5fc bytes) 2024-09-18T01:10:36.153 INFO:teuthology.orchestra.run.smithi120.stdout:5204 trunc from 0xbd172f4 to 0x2d1de16 2024-09-18T01:10:36.177 INFO:teuthology.orchestra.run.smithi120.stdout:5206 write 0x702cfb6 thru 0x7031fca (0x5015 bytes) 2024-09-18T01:10:36.185 INFO:teuthology.orchestra.run.smithi120.stdout:5207 trunc from 0x7031fcb to 0xaacc190 2024-09-18T01:10:36.190 INFO:teuthology.orchestra.run.smithi120.stdout:5208 read 0x6d3f2c4 thru 0x6d3f89e (0x5db bytes) 2024-09-18T01:10:36.191 INFO:teuthology.orchestra.run.smithi120.stdout:5209 write 0x187326c thru 0x1880e63 (0xdbf8 bytes) 2024-09-18T01:10:36.192 INFO:teuthology.orchestra.run.smithi120.stdout:5211 write 0xa3feb91 thru 0xa4035ed (0x4a5d bytes) 2024-09-18T01:10:36.200 INFO:teuthology.orchestra.run.smithi120.stdout:5212 trunc from 0xaacc190 to 0xe20de7b 2024-09-18T01:10:36.207 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.207 INFO:teuthology.orchestra.run.smithi120.stdout:5214 writesame 0xcd9809 thru 0xce6ed9 data_size 0x17(0xd6d1 bytes) 2024-09-18T01:10:36.210 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.210 INFO:teuthology.orchestra.run.smithi120.stdout:5216 write 0xc742cae thru 0xc74a1a8 (0x74fb bytes) 2024-09-18T01:10:36.214 INFO:teuthology.orchestra.run.smithi120.stdout:5217 writesame 0x90ca7b9 thru 0x90d113f data_size 0x5(0x6987 bytes) 2024-09-18T01:10:36.220 INFO:teuthology.orchestra.run.smithi120.stdout:5218 write 0x9dbac25 thru 0x9dbbc2f (0x100b bytes) 2024-09-18T01:10:36.220 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:36.219+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:36.221 INFO:teuthology.orchestra.run.smithi120.stdout:5219 punch from 0x1568148 to 0x1576d74, (0xec2c bytes) 2024-09-18T01:10:36.221 INFO:teuthology.orchestra.run.smithi120.stdout:5221 write 0x11eb686 thru 0x11f6bf3 (0xb56e bytes) 2024-09-18T01:10:36.223 INFO:teuthology.orchestra.run.smithi120.stdout:5222 punch from 0x3fe0edf to 0x3fe6b6c, (0x5c8d bytes) 2024-09-18T01:10:36.229 INFO:teuthology.orchestra.run.smithi120.stdout:5223 write 0xe55e290 thru 0xe56c97b (0xe6ec bytes) 2024-09-18T01:10:36.232 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.247 INFO:teuthology.orchestra.run.smithi120.stdout:5227 trunc from 0xe56c97c to 0x24692ea 2024-09-18T01:10:36.268 INFO:teuthology.orchestra.run.smithi120.stdout:5228 read 0x9c4356 thru 0x9d2fc6 (0xec71 bytes) 2024-09-18T01:10:36.274 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.275 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.284 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:36.284 INFO:teuthology.orchestra.run.smithi120.stdout:5229 write 0xad21b0e thru 0xad237be (0x1cb1 bytes) 2024-09-18T01:10:36.288 INFO:teuthology.orchestra.run.smithi120.stdout:5230 write 0x84616c5 thru 0x8462941 (0x127d bytes) 2024-09-18T01:10:36.291 INFO:teuthology.orchestra.run.smithi120.stdout:5231 write 0x65a76da thru 0x65a7f36 (0x85d bytes) 2024-09-18T01:10:36.294 INFO:teuthology.orchestra.run.smithi120.stdout:5232 write 0x2c9784 thru 0x2cbe56 (0x26d3 bytes) 2024-09-18T01:10:36.295 INFO:teuthology.orchestra.run.smithi120.stdout:5233 write 0x274697d thru 0x274fc99 (0x931d bytes) 2024-09-18T01:10:36.297 INFO:teuthology.orchestra.run.smithi120.stdout:5235 read 0x55d900f thru 0x55df138 (0x612a bytes) 2024-09-18T01:10:36.297 INFO:teuthology.orchestra.run.smithi120.stdout:5236 read 0x5fed9e4 thru 0x5fee825 (0xe42 bytes) 2024-09-18T01:10:36.298 INFO:teuthology.orchestra.run.smithi120.stdout:5238 write 0xdf60c1 thru 0xe003e9 (0xa329 bytes) 2024-09-18T01:10:36.309 INFO:teuthology.orchestra.run.smithi120.stdout:5239 trunc from 0xad237bf to 0x418393c 2024-09-18T01:10:36.325 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:36.324+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:36.336 INFO:teuthology.orchestra.run.smithi120.stdout:5240 writesame 0x93ff81b thru 0x9403b22 data_size 0x58(0x4308 bytes) 2024-09-18T01:10:36.338 INFO:teuthology.orchestra.run.smithi120.stdout:5241 trunc from 0x9403b23 to 0xb2a0cb2 2024-09-18T01:10:36.349 INFO:teuthology.orchestra.run.smithi120.stdout:5242 trunc from 0xb2a0cb2 to 0x79d21ef 2024-09-18T01:10:36.355 INFO:teuthology.orchestra.run.smithi120.stdout:5243 write 0x1831e6 thru 0x191984 (0xe79f bytes) 2024-09-18T01:10:36.360 INFO:teuthology.orchestra.run.smithi120.stdout:5244 trunc from 0x79d21ef to 0x538741a 2024-09-18T01:10:36.366 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.375 INFO:teuthology.orchestra.run.smithi120.stdout:5246 writesame 0x859a067 thru 0x859d33b data_size 0xd(0x32d5 bytes) 2024-09-18T01:10:36.375 INFO:teuthology.orchestra.run.smithi120.stdout:5247 write 0x4114de1 thru 0x411cff7 (0x8217 bytes) 2024-09-18T01:10:36.387 INFO:teuthology.orchestra.run.smithi120.stdout:5249 trunc from 0x859d33c to 0x111cf8b 2024-09-18T01:10:36.413 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:10:36.413 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:36.413 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 200, 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:10:35.654311+0000", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:07:11.886968+0000", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:10:20.585169+0000", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5804032, 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:10:36.414 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "jewel", 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:10:36.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:36.416 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:36.417 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:36.418 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:36.419 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:36.420 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:10:36.421 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:10:36.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:10:36.423 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:36.424 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:36.425 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:36.426 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:10:36.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 2, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 2, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.75, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:10:36.429 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:10:36.430 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "200", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 89, 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 200, 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:10:36.431 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:10:36.432 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:10:36.433 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.4900000095367432, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.4900000095367432, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.309999942779541, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.309999942779541, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:10:36.434 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.435 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 166, 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 188, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.441 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.443 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:10:36.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:10:36.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 162, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.448 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.450 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.450 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.450 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.451 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 185, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.454 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.458 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.459 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 188, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:10:36.460 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:10:36.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:10:36.462 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:10:36.464 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.465 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 185, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.466 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 163, 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.475 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.476 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 164, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:10:36.479 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:10:36.481 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.482 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:10:36.483 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 65536, 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:10:36.485 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.486 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:10:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [], 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:10:36.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 85, 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 87, 2024-09-18T01:10:36.490 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 89, 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:10:36.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 89, 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:10:36.492 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:10:36.493 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:10:36.493 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:10:36.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:36.493 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:36.494 INFO:teuthology.orchestra.run.smithi120.stdout:5250 write 0x94910b3 thru 0x949f333 (0xe281 bytes) 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi120.stdout:5251 punch from 0x14f0ea to 0x15306f, (0x3f85 bytes) 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi120.stdout:5252 read 0x79c073 thru 0x7a76b7 (0xb645 bytes) 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi120.stdout:5254 read 0x143450e thru 0x143b0a5 (0x6b98 bytes) 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi120.stdout:5255 trunc from 0x949f334 to 0x7167490 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.495 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:36.497 INFO:tasks.thrashosds.thrasher:j is {'epoch': 200, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:10:35.654311+0000', 'last_up_change': '2024-09-18T01:07:11.886968+0000', 'last_in_change': '2024-09-18T01:10:20.585169+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 89, 'snap_epoch': 200, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 188, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 162, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 185, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 188, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 185, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 85, 'length': 1}, {'begin': 87, 'length': 1}, {'begin': 89, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 89, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:10:36.497 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:10:36.498 INFO:teuthology.orchestra.run.smithi120.stdout:5256 trunc from 0x7167490 to 0xcce2b64 2024-09-18T01:10:36.498 INFO:teuthology.orchestra.run.smithi120.stdout:5257 punch from 0x7e4afa6 to 0x7e4e31c, (0x3376 bytes) 2024-09-18T01:10:36.498 INFO:teuthology.orchestra.run.smithi120.stdout:5258 read 0x961e4a4 thru 0x961f022 (0xb7f bytes) 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5259 read 0xb3cb6c7 thru 0xb3d6228 (0xab62 bytes) 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5261 write 0x140ac99 thru 0x1415259 (0xa5c1 bytes) 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5263 punch from 0x32e3902 to 0x32f14c7, (0xdbc5 bytes) 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5265 write 0x5b7b659 thru 0x5b7ea53 (0x33fb bytes) 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5271 trunc from 0xcce2b64 to 0x46efe2e 2024-09-18T01:10:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:5272 write 0x3d5cb9c thru 0x3d6173b (0x4ba0 bytes) 2024-09-18T01:10:36.499 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:36.493+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:36.500 INFO:teuthology.orchestra.run.smithi120.stdout:5273 write 0x9a25dc0 thru 0x9a2d51e (0x775f bytes) 2024-09-18T01:10:36.501 INFO:teuthology.orchestra.run.smithi120.stdout:5275 write 0x3c192ba thru 0x3c1dcbc (0x4a03 bytes) 2024-09-18T01:10:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:5276 trunc from 0x9a2d51f to 0x7a9d4e1 2024-09-18T01:10:36.516 INFO:teuthology.orchestra.run.smithi120.stdout:5277 read 0x656c70b thru 0x656e586 (0x1e7c bytes) 2024-09-18T01:10:36.516 INFO:teuthology.orchestra.run.smithi120.stdout:5279 write 0x47c6c79 thru 0x47d626e (0xf5f6 bytes) 2024-09-18T01:10:36.521 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.521 INFO:teuthology.orchestra.run.smithi120.stdout:5282 write 0x3456505 thru 0x3462ba2 (0xc69e bytes) 2024-09-18T01:10:36.523 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:36.523 INFO:teuthology.orchestra.run.smithi120.stdout:5284 read 0x17b0c6 thru 0x17d4cf (0x240a bytes) 2024-09-18T01:10:36.526 INFO:teuthology.orchestra.run.smithi120.stdout:5286 trunc from 0x7a9d4e1 to 0x611e44f 2024-09-18T01:10:36.533 INFO:teuthology.orchestra.run.smithi120.stdout:5290 clone 44 order 25 su 8388608 sc 12 2024-09-18T01:10:36.595 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:36.594+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:36.603 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.604 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.612 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:36.667 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:10:36.697 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:36.696+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:36.760 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.760 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:36.797 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:10:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:10:36.799 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:36.796+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:36.809 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-18T01:10:36.756829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1398983664,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":244990,"num_read_kb":1212734,"num_write":30825,"num_write_kb":830785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1725,"num_bytes_recovered":217255114,"num_keys_recovered":198,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20419,"ondisk_log_size":20419,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":1175228,"kb_used_data":747288,"kb_used_omap":10,"kb_used_meta":427765,"kb_avail":659427652,"statfs":{"total":676457349120,"available":675253915648,"internally_reserved":0,"allocated":765222912,"data_stored":750675902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":438031467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59901931,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":3432,"num_read_kb":13967,"num_write":710,"num_write_kb":6089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000889"},"pg_stats":[{"pgid":"3.1f","version":"196'455","reported_seq":6738,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.657653+0000","last_change":"2024-09-18T01:10:32.657966+0000","last_active":"2024-09-18T01:10:32.657653+0000","last_peered":"2024-09-18T01:10:32.657653+0000","last_clean":"2024-09-18T01:10:32.657653+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:10:32.657653+0000","last_undegraded":"2024-09-18T01:10:32.657653+0000","last_fullsized":"2024-09-18T01:10:32.657653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:34.982509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073061600000000001,"stat_sum":{"num_bytes":46904373,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":6240,"num_read_kb":39217,"num_write":565,"num_write_kb":34035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.1e","version":"197'652","reported_seq":3924,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.617988+0000","last_change":"2024-09-18T01:10:32.654613+0000","last_active":"2024-09-18T01:10:34.617988+0000","last_peered":"2024-09-18T01:10:34.617988+0000","last_clean":"2024-09-18T01:10:34.617988+0000","last_became_active":"2024-09-18T01:07:28.974548+0000","last_became_peered":"2024-09-18T01:07:28.974548+0000","last_unstale":"2024-09-18T01:10:34.617988+0000","last_undegraded":"2024-09-18T01:10:34.617988+0000","last_fullsized":"2024-09-18T01:10:34.617988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:01.100766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674300000000004,"stat_sum":{"num_bytes":23862593,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4590,"num_read_kb":33168,"num_write":1055,"num_write_kb":25657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.1d","version":"196'562","reported_seq":4084,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.660417+0000","last_change":"2024-09-18T01:10:32.660630+0000","last_active":"2024-09-18T01:10:32.660417+0000","last_peered":"2024-09-18T01:10:32.660417+0000","last_clean":"2024-09-18T01:10:32.660417+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:10:32.660417+0000","last_undegraded":"2024-09-18T01:10:32.660417+0000","last_fullsized":"2024-09-18T01:10:32.660417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:34:45.109545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060881900000000003,"stat_sum":{"num_bytes":35218786,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4062,"num_read_kb":35131,"num_write":892,"num_write_kb":38637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.1c","version":"196'521","reported_seq":5208,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.660205+0000","last_change":"2024-09-18T01:10:32.660370+0000","last_active":"2024-09-18T01:10:32.660205+0000","last_peered":"2024-09-18T01:10:32.660205+0000","last_clean":"2024-09-18T01:10:32.660205+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:10:32.660205+0000","last_undegraded":"2024-09-18T01:10:32.660205+0000","last_fullsized":"2024-09-18T01:10:32.660205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:51.502766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274599999999999,"stat_sum":{"num_bytes":38972986,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5013,"num_read_kb":38501,"num_write":719,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.1b","version":"196'1102","reported_seq":6761,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.657580+0000","last_change":"2024-09-18T01:10:32.657841+0000","last_active":"2024-09-18T01:10:32.657580+0000","last_peered":"2024-09-18T01:10:32.657580+0000","last_clean":"2024-09-18T01:10:32.657580+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:10:32.657580+0000","last_undegraded":"2024-09-18T01:10:32.657580+0000","last_fullsized":"2024-09-18T01:10:32.657580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:21.524293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079622799999999995,"stat_sum":{"num_bytes":36511800,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":9430,"num_read_kb":41719,"num_write":1653,"num_write_kb":30276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.1a","version":"199'394","reported_seq":6550,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.653454+0000","last_change":"2024-09-18T01:10:32.648345+0000","last_active":"2024-09-18T01:10:34.653454+0000","last_peered":"2024-09-18T01:10:34.653454+0000","last_clean":"2024-09-18T01:10:34.653454+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:10:34.653454+0000","last_undegraded":"2024-09-18T01:10:34.653454+0000","last_fullsized":"2024-09-18T01:10:34.653454+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.417355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058046400000000005,"stat_sum":{"num_bytes":31123257,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5715,"num_read_kb":36687,"num_write":542,"num_write_kb":25895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.19","version":"196'436","reported_seq":8388,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.647991+0000","last_change":"2024-09-18T01:10:32.648233+0000","last_active":"2024-09-18T01:10:32.647991+0000","last_peered":"2024-09-18T01:10:32.647991+0000","last_clean":"2024-09-18T01:10:32.647991+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:10:32.647991+0000","last_undegraded":"2024-09-18T01:10:32.647991+0000","last_fullsized":"2024-09-18T01:10:32.647991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:12.833313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041487000000000002,"stat_sum":{"num_bytes":52945432,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":7813,"num_read_kb":37720,"num_write":564,"num_write_kb":24167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.18","version":"199'466","reported_seq":12710,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.619559+0000","last_change":"2024-09-18T01:10:32.655824+0000","last_active":"2024-09-18T01:10:34.619559+0000","last_peered":"2024-09-18T01:10:34.619559+0000","last_clean":"2024-09-18T01:10:34.619559+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:10:34.619559+0000","last_undegraded":"2024-09-18T01:10:34.619559+0000","last_fullsized":"2024-09-18T01:10:34.619559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:15:04.531088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001752733,"stat_sum":{"num_bytes":57666059,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":12506,"num_read_kb":57364,"num_write":587,"num_write_kb":24252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.17","version":"196'1104","reported_seq":7147,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.657338+0000","last_change":"2024-09-18T01:10:32.657525+0000","last_active":"2024-09-18T01:10:32.657338+0000","last_peered":"2024-09-18T01:10:32.657338+0000","last_clean":"2024-09-18T01:10:32.657338+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:10:32.657338+0000","last_undegraded":"2024-09-18T01:10:32.657338+0000","last_fullsized":"2024-09-18T01:10:32.657338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1104,"log_dups_size":0,"ondisk_log_size":1104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:27.766603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058975900000000003,"stat_sum":{"num_bytes":44689190,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":8957,"num_read_kb":40190,"num_write":1826,"num_write_kb":27486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.16","version":"196'594","reported_seq":4060,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.660260+0000","last_change":"2024-09-18T01:10:32.660431+0000","last_active":"2024-09-18T01:10:32.660260+0000","last_peered":"2024-09-18T01:10:32.660260+0000","last_clean":"2024-09-18T01:10:32.660260+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:10:32.660260+0000","last_undegraded":"2024-09-18T01:10:32.660260+0000","last_fullsized":"2024-09-18T01:10:32.660260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:59:49.504808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042348899999999998,"stat_sum":{"num_bytes":33538223,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":4294,"num_read_kb":21797,"num_write":857,"num_write_kb":19290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.15","version":"196'727","reported_seq":8731,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.734063+0000","last_change":"2024-09-18T01:10:32.734252+0000","last_active":"2024-09-18T01:10:32.734063+0000","last_peered":"2024-09-18T01:10:32.734063+0000","last_clean":"2024-09-18T01:10:32.734063+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:10:32.734063+0000","last_undegraded":"2024-09-18T01:10:32.734063+0000","last_fullsized":"2024-09-18T01:10:32.734063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:05.480004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076994900000000002,"stat_sum":{"num_bytes":47214454,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":9346,"num_read_kb":57680,"num_write":1127,"num_write_kb":29463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.14","version":"199'593","reported_seq":5936,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.617452+0000","last_change":"2024-09-18T01:10:32.648579+0000","last_active":"2024-09-18T01:10:34.617452+0000","last_peered":"2024-09-18T01:10:34.617452+0000","last_clean":"2024-09-18T01:10:34.617452+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:10:34.617452+0000","last_undegraded":"2024-09-18T01:10:34.617452+0000","last_fullsized":"2024-09-18T01:10:34.617452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:39:51.120973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065251800000000004,"stat_sum":{"num_bytes":63520435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":6357,"num_read_kb":23335,"num_write":897,"num_write_kb":16015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":329,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.711209+0000","last_change":"2024-09-18T01:09:57.581850+0000","last_active":"2024-09-18T01:10:32.711209+0000","last_peered":"2024-09-18T01:10:32.711209+0000","last_clean":"2024-09-18T01:10:32.711209+0000","last_became_active":"2024-09-18T01:09:57.581537+0000","last_became_peered":"2024-09-18T01:09:57.581537+0000","last_unstale":"2024-09-18T01:10:32.711209+0000","last_undegraded":"2024-09-18T01:10:32.711209+0000","last_fullsized":"2024-09-18T01:10:32.711209+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:47.001712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"196'481","reported_seq":3778,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.657281+0000","last_change":"2024-09-18T01:10:32.657480+0000","last_active":"2024-09-18T01:10:32.657281+0000","last_peered":"2024-09-18T01:10:32.657281+0000","last_clean":"2024-09-18T01:10:32.657281+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:10:32.657281+0000","last_undegraded":"2024-09-18T01:10:32.657281+0000","last_fullsized":"2024-09-18T01:10:32.657281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:25.001143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054478999999999997,"stat_sum":{"num_bytes":18303193,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":3943,"num_read_kb":25807,"num_write":707,"num_write_kb":21678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":372,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.647617+0000","last_change":"2024-09-18T01:07:20.954875+0000","last_active":"2024-09-18T01:10:32.647617+0000","last_peered":"2024-09-18T01:10:32.647617+0000","last_clean":"2024-09-18T01:10:32.647617+0000","last_became_active":"2024-09-18T01:07:18.939162+0000","last_became_peered":"2024-09-18T01:07:18.939162+0000","last_unstale":"2024-09-18T01:10:32.647617+0000","last_undegraded":"2024-09-18T01:10:32.647617+0000","last_fullsized":"2024-09-18T01:10:32.647617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:46.459634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"196'394","reported_seq":6882,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.615716+0000","last_change":"2024-09-18T01:10:32.654929+0000","last_active":"2024-09-18T01:10:34.615716+0000","last_peered":"2024-09-18T01:10:34.615716+0000","last_clean":"2024-09-18T01:10:34.615716+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:10:34.615716+0000","last_undegraded":"2024-09-18T01:10:34.615716+0000","last_fullsized":"2024-09-18T01:10:34.615716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:58.967657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099742600000000004,"stat_sum":{"num_bytes":32482664,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":6440,"num_read_kb":29363,"num_write":660,"num_write_kb":23592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":371,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733343+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:10:32.733343+0000","last_peered":"2024-09-18T01:10:32.733343+0000","last_clean":"2024-09-18T01:10:32.733343+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:10:32.733343+0000","last_undegraded":"2024-09-18T01:10:32.733343+0000","last_fullsized":"2024-09-18T01:10:32.733343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"197'343","reported_seq":6138,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.648559+0000","last_change":"2024-09-18T01:10:32.648559+0000","last_active":"2024-09-18T01:10:32.648559+0000","last_peered":"2024-09-18T01:10:32.648559+0000","last_clean":"2024-09-18T01:10:32.648559+0000","last_became_active":"2024-09-18T01:10:24.570292+0000","last_became_peered":"2024-09-18T01:10:24.570292+0000","last_unstale":"2024-09-18T01:10:32.648559+0000","last_undegraded":"2024-09-18T01:10:32.648559+0000","last_fullsized":"2024-09-18T01:10:32.648559+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:56.657841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086592799999999997,"stat_sum":{"num_bytes":43111602,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4959,"num_read_kb":46583,"num_write":424,"num_write_kb":25964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":36735962,"num_keys_recovered":66,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":372,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.647682+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:10:32.647682+0000","last_peered":"2024-09-18T01:10:32.647682+0000","last_clean":"2024-09-18T01:10:32.647682+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:10:32.647682+0000","last_undegraded":"2024-09-18T01:10:32.647682+0000","last_fullsized":"2024-09-18T01:10:32.647682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"196'830","reported_seq":12107,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.660669+0000","last_change":"2024-09-18T01:10:32.660871+0000","last_active":"2024-09-18T01:10:32.660669+0000","last_peered":"2024-09-18T01:10:32.660669+0000","last_clean":"2024-09-18T01:10:32.660669+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:10:32.660669+0000","last_undegraded":"2024-09-18T01:10:32.660669+0000","last_fullsized":"2024-09-18T01:10:32.660669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:55.488668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829724,"stat_sum":{"num_bytes":61667426,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":13559,"num_read_kb":62335,"num_write":1339,"num_write_kb":38710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.3","version":"197'438","reported_seq":8349,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.613801+0000","last_change":"2024-09-18T01:10:32.648383+0000","last_active":"2024-09-18T01:10:34.613801+0000","last_peered":"2024-09-18T01:10:34.613801+0000","last_clean":"2024-09-18T01:10:34.613801+0000","last_became_active":"2024-09-18T01:07:28.973632+0000","last_became_peered":"2024-09-18T01:07:28.973632+0000","last_unstale":"2024-09-18T01:10:34.613801+0000","last_undegraded":"2024-09-18T01:10:34.613801+0000","last_fullsized":"2024-09-18T01:10:34.613801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:27.865930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056037800000000005,"stat_sum":{"num_bytes":34091673,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":8043,"num_read_kb":43214,"num_write":648,"num_write_kb":23835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":373,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.659775+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:10:32.659775+0000","last_peered":"2024-09-18T01:10:32.659775+0000","last_clean":"2024-09-18T01:10:32.659775+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:10:32.659775+0000","last_undegraded":"2024-09-18T01:10:32.659775+0000","last_fullsized":"2024-09-18T01:10:32.659775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"196'467","reported_seq":12504,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.647926+0000","last_change":"2024-09-18T01:10:32.648187+0000","last_active":"2024-09-18T01:10:32.647926+0000","last_peered":"2024-09-18T01:10:32.647926+0000","last_clean":"2024-09-18T01:10:32.647926+0000","last_became_active":"2024-09-18T01:10:24.573478+0000","last_became_peered":"2024-09-18T01:10:24.573478+0000","last_unstale":"2024-09-18T01:10:32.647926+0000","last_undegraded":"2024-09-18T01:10:32.647926+0000","last_fullsized":"2024-09-18T01:10:32.647926+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:59.853251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000454287,"stat_sum":{"num_bytes":54252837,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":11908,"num_read_kb":57243,"num_write":787,"num_write_kb":27953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":38165252,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.569366+0000","last_change":"2024-09-18T01:10:21.608928+0000","last_active":"2024-09-18T01:10:34.569366+0000","last_peered":"2024-09-18T01:10:34.569366+0000","last_clean":"2024-09-18T01:10:34.569366+0000","last_became_active":"2024-09-18T01:10:21.608656+0000","last_became_peered":"2024-09-18T01:10:21.608656+0000","last_unstale":"2024-09-18T01:10:34.569366+0000","last_undegraded":"2024-09-18T01:10:34.569366+0000","last_fullsized":"2024-09-18T01:10:34.569366+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:30.706634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"196'591","reported_seq":14471,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.657039+0000","last_change":"2024-09-18T01:10:32.657220+0000","last_active":"2024-09-18T01:10:32.657039+0000","last_peered":"2024-09-18T01:10:32.657039+0000","last_clean":"2024-09-18T01:10:32.657039+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:10:32.657039+0000","last_undegraded":"2024-09-18T01:10:32.657039+0000","last_fullsized":"2024-09-18T01:10:32.657039+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:07.435596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045971800000000002,"stat_sum":{"num_bytes":92150428,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":14288,"num_read_kb":42163,"num_write":750,"num_write_kb":34196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":371,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.732857+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:10:32.732857+0000","last_peered":"2024-09-18T01:10:32.732857+0000","last_clean":"2024-09-18T01:10:32.732857+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:10:32.732857+0000","last_undegraded":"2024-09-18T01:10:32.732857+0000","last_fullsized":"2024-09-18T01:10:32.732857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"199'414","reported_seq":7783,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.617901+0000","last_change":"2024-09-18T01:10:32.654289+0000","last_active":"2024-09-18T01:10:34.617901+0000","last_peered":"2024-09-18T01:10:34.617901+0000","last_clean":"2024-09-18T01:10:34.617901+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:10:34.617901+0000","last_undegraded":"2024-09-18T01:10:34.617901+0000","last_fullsized":"2024-09-18T01:10:34.617901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:01.629520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000457644,"stat_sum":{"num_bytes":57488243,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":7057,"num_read_kb":32798,"num_write":507,"num_write_kb":33138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":399,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.659721+0000","last_change":"2024-09-18T01:10:21.608402+0000","last_active":"2024-09-18T01:10:32.659721+0000","last_peered":"2024-09-18T01:10:32.659721+0000","last_clean":"2024-09-18T01:10:32.659721+0000","last_became_active":"2024-09-18T01:10:21.606625+0000","last_became_peered":"2024-09-18T01:10:21.606625+0000","last_unstale":"2024-09-18T01:10:32.659721+0000","last_undegraded":"2024-09-18T01:10:32.659721+0000","last_fullsized":"2024-09-18T01:10:32.659721+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:55.887889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":523,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.732987+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:10:32.732987+0000","last_peered":"2024-09-18T01:10:32.732987+0000","last_clean":"2024-09-18T01:10:32.732987+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:10:32.732987+0000","last_undegraded":"2024-09-18T01:10:32.732987+0000","last_fullsized":"2024-09-18T01:10:32.732987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"199'1073","reported_seq":4220,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.666657+0000","last_change":"2024-09-18T01:10:32.654911+0000","last_active":"2024-09-18T01:10:34.666657+0000","last_peered":"2024-09-18T01:10:34.666657+0000","last_clean":"2024-09-18T01:10:34.666657+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:10:34.666657+0000","last_undegraded":"2024-09-18T01:10:34.666657+0000","last_fullsized":"2024-09-18T01:10:34.666657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:52.808390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010083430000000001,"stat_sum":{"num_bytes":24260185,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":6102,"num_read_kb":20359,"num_write":1759,"num_write_kb":16727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.9","version":"197'651","reported_seq":11870,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.607089+0000","last_change":"2024-09-18T01:10:32.648123+0000","last_active":"2024-09-18T01:10:34.607089+0000","last_peered":"2024-09-18T01:10:34.607089+0000","last_clean":"2024-09-18T01:10:34.607089+0000","last_became_active":"2024-09-18T01:10:24.575146+0000","last_became_peered":"2024-09-18T01:10:24.575146+0000","last_unstale":"2024-09-18T01:10:34.607089+0000","last_undegraded":"2024-09-18T01:10:34.607089+0000","last_fullsized":"2024-09-18T01:10:34.607089+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:26:37.502219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479094,"stat_sum":{"num_bytes":47736776,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":11927,"num_read_kb":43441,"num_write":958,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":397,"num_bytes_recovered":45896415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.a","version":"196'421","reported_seq":10449,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.711476+0000","last_change":"2024-09-18T01:10:32.711673+0000","last_active":"2024-09-18T01:10:32.711476+0000","last_peered":"2024-09-18T01:10:32.711476+0000","last_clean":"2024-09-18T01:10:32.711476+0000","last_became_active":"2024-09-18T01:08:30.552283+0000","last_became_peered":"2024-09-18T01:08:30.552283+0000","last_unstale":"2024-09-18T01:10:32.711476+0000","last_undegraded":"2024-09-18T01:10:32.711476+0000","last_fullsized":"2024-09-18T01:10:32.711476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:08.556196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038612199999999997,"stat_sum":{"num_bytes":66379922,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":10104,"num_read_kb":42729,"num_write":557,"num_write_kb":22649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.b","version":"197'598","reported_seq":4470,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T01:10:34.615709+0000","last_change":"2024-09-18T01:10:32.654287+0000","last_active":"2024-09-18T01:10:34.615709+0000","last_peered":"2024-09-18T01:10:34.615709+0000","last_clean":"2024-09-18T01:10:34.615709+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:10:34.615709+0000","last_undegraded":"2024-09-18T01:10:34.615709+0000","last_fullsized":"2024-09-18T01:10:34.615709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:08.607828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048694899999999998,"stat_sum":{"num_bytes":26847778,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":4650,"num_read_kb":32528,"num_write":851,"num_write_kb":23840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.c","version":"196'928","reported_seq":6451,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.660473+0000","last_change":"2024-09-18T01:10:32.660709+0000","last_active":"2024-09-18T01:10:32.660473+0000","last_peered":"2024-09-18T01:10:32.660473+0000","last_clean":"2024-09-18T01:10:32.660473+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:10:32.660473+0000","last_undegraded":"2024-09-18T01:10:32.660473+0000","last_fullsized":"2024-09-18T01:10:32.660473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:16.174526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069816300000000002,"stat_sum":{"num_bytes":24518270,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":7707,"num_read_kb":32364,"num_write":1590,"num_write_kb":22454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.d","version":"196'572","reported_seq":13423,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733738+0000","last_change":"2024-09-18T01:10:32.733917+0000","last_active":"2024-09-18T01:10:32.733738+0000","last_peered":"2024-09-18T01:10:32.733738+0000","last_clean":"2024-09-18T01:10:32.733738+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:10:32.733738+0000","last_undegraded":"2024-09-18T01:10:32.733738+0000","last_fullsized":"2024-09-18T01:10:32.733738+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:41.343587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070084699999999999,"stat_sum":{"num_bytes":65315435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":13419,"num_read_kb":51767,"num_write":835,"num_write_kb":26738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.e","version":"196'1109","reported_seq":6999,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733802+0000","last_change":"2024-09-18T01:10:32.734000+0000","last_active":"2024-09-18T01:10:32.733802+0000","last_peered":"2024-09-18T01:10:32.733802+0000","last_clean":"2024-09-18T01:10:32.733802+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:10:32.733802+0000","last_undegraded":"2024-09-18T01:10:32.733802+0000","last_fullsized":"2024-09-18T01:10:32.733802+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1109,"log_dups_size":0,"ondisk_log_size":1109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:13:03.734003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072993699999999995,"stat_sum":{"num_bytes":43420613,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":8737,"num_read_kb":54976,"num_write":1826,"num_write_kb":37771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.f","version":"196'359","reported_seq":5366,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733972+0000","last_change":"2024-09-18T01:10:32.734165+0000","last_active":"2024-09-18T01:10:32.733972+0000","last_peered":"2024-09-18T01:10:32.733972+0000","last_clean":"2024-09-18T01:10:32.733972+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:10:32.733972+0000","last_undegraded":"2024-09-18T01:10:32.733972+0000","last_fullsized":"2024-09-18T01:10:32.733972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:41.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083490300000000001,"stat_sum":{"num_bytes":49068138,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4686,"num_read_kb":29592,"num_write":504,"num_write_kb":16854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.10","version":"196'473","reported_seq":4002,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.711541+0000","last_change":"2024-09-18T01:10:32.711726+0000","last_active":"2024-09-18T01:10:32.711541+0000","last_peered":"2024-09-18T01:10:32.711541+0000","last_clean":"2024-09-18T01:10:32.711541+0000","last_became_active":"2024-09-18T01:08:30.554232+0000","last_became_peered":"2024-09-18T01:08:30.554232+0000","last_unstale":"2024-09-18T01:10:32.711541+0000","last_undegraded":"2024-09-18T01:10:32.711541+0000","last_fullsized":"2024-09-18T01:10:32.711541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:37.591414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033269899999999999,"stat_sum":{"num_bytes":24070002,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":3788,"num_read_kb":28267,"num_write":650,"num_write_kb":22403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.11","version":"196'549","reported_seq":4326,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733318+0000","last_change":"2024-09-18T01:10:32.733518+0000","last_active":"2024-09-18T01:10:32.733318+0000","last_peered":"2024-09-18T01:10:32.733318+0000","last_clean":"2024-09-18T01:10:32.733318+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:10:32.733318+0000","last_undegraded":"2024-09-18T01:10:32.733318+0000","last_fullsized":"2024-09-18T01:10:32.733318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:22.700204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037938499999999999,"stat_sum":{"num_bytes":64567008,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4686,"num_read_kb":27308,"num_write":738,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.12","version":"196'1233","reported_seq":5866,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.656974+0000","last_change":"2024-09-18T01:10:32.657237+0000","last_active":"2024-09-18T01:10:32.656974+0000","last_peered":"2024-09-18T01:10:32.656974+0000","last_clean":"2024-09-18T01:10:32.656974+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:10:32.656974+0000","last_undegraded":"2024-09-18T01:10:32.656974+0000","last_fullsized":"2024-09-18T01:10:32.656974+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1233,"log_dups_size":0,"ondisk_log_size":1233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:47.496866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056923300000000004,"stat_sum":{"num_bytes":23240495,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":7921,"num_read_kb":28350,"num_write":2140,"num_write_kb":19934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]},{"pgid":"3.13","version":"196'769","reported_seq":5425,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T01:10:32.733383+0000","last_change":"2024-09-18T01:10:32.733531+0000","last_active":"2024-09-18T01:10:32.733383+0000","last_peered":"2024-09-18T01:10:32.733383+0000","last_clean":"2024-09-18T01:10:32.733383+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:10:32.733383+0000","last_undegraded":"2024-09-18T01:10:32.733383+0000","last_fullsized":"2024-09-18T01:10:32.733383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:33.117762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037104500000000002,"stat_sum":{"num_bytes":33253001,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":6626,"num_read_kb":18937,"num_write":1150,"num_write_kb":14774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1398393277,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":244873,"num_read_kb":1212633,"num_write":30664,"num_write_kb":828367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1722,"num_bytes_recovered":217255114,"num_keys_recovered":198,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":788639744,"data_stored":782217922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20299,"ondisk_log_size":20299,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185528,"kb_used_data":109916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94186312,"statfs":{"total":96636764160,"available":96446783488,"internally_reserved":0,"allocated":112553984,"data_stored":110523447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88624,"kb_used_data":49116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94283216,"statfs":{"total":96636764160,"available":96546013184,"internally_reserved":0,"allocated":50294784,"data_stored":48551119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206208,"kb_used_data":134764,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94165632,"statfs":{"total":96636764160,"available":96425607168,"internally_reserved":0,"allocated":137998336,"data_stored":135834205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509484,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100904,"kb_used_data":73876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94270936,"statfs":{"total":96636764160,"available":96533438464,"internally_reserved":0,"allocated":75649024,"data_stored":73813734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188512,"kb_used_data":112904,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94183328,"statfs":{"total":96636764160,"available":96443727872,"internally_reserved":0,"allocated":115613696,"data_stored":113359956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208708,"kb_used_data":145584,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94163132,"statfs":{"total":96636764160,"available":96423047168,"internally_reserved":0,"allocated":149078016,"data_stored":146752920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196744,"kb_used_data":121128,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94175096,"statfs":{"total":96636764160,"available":96435298304,"internally_reserved":0,"allocated":124035072,"data_stored":121840521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112660480,"data_stored":111622152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73285632,"data_stored":72666279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41070592,"data_stored":40669431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":121769984,"data_stored":120797205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":146681856,"data_stored":145586779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":135634944,"data_stored":134686750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":47935488,"data_stored":47403683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":109600768,"data_stored":108785643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:10:36.812 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [4, 6, 5, 1] 2024-09-18T01:10:36.812 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '4', '6', '5', '1'] 2024-09-18T01:10:36.812 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 4 6 5 1 2024-09-18T01:10:36.873 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:36.874 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:36.883 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:36.897 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:36.896+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:36.999 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:36.998+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:37.027 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.028 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:37.121 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:37.120+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:37.180 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.181 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:37.231 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:37.230+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:37.301 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:37.300+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:37.333 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.333 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.341 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:37.402 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:37.402+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:37.443 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.443 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.451 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:37.503 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:37.502+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:37.555 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.555 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:37.605 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:37.604+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:37.668 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.3 pg_upmap_items mapping to [4->6,5->1] 2024-09-18T01:10:37.705 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:37.704+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:37.716 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.716 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:37.806 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:37.805+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:37.832 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:37.871 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:37.871 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:37.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:37.907 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:37.906+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:37.955 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:37.974 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:37.990 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:38.006 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:38.007 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:38.005+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:38.024 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:38.024 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.030 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.367505080000001, 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:38.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.367450855000001, 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:38.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:38.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:38.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:38.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.367418396999994, 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:38.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:38.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:38.045 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.367287133999994, 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:38.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:38.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.366817147999996, 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:38.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.365654547999995, 2024-09-18T01:10:38.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:38.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.365547265999993, 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.061 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:38.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.365465607000004, 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:38.066 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:38.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.364950218999994, 2024-09-18T01:10:38.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:38.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.364130981000002, 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:38.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.790865895000003, 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:38.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.624007003000003, 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:38.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.596161615, 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:38.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.596141316000001, 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:38.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:38.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:38.104 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.596121052999997, 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:38.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.5961012, 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:38.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:38.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.116 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.596082043000003, 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:38.122 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.212091616999999, 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:38.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:38.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:38.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.152132754, 2024-09-18T01:10:38.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:38.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.367629308000001, 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.134 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:38.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:38.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:38.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.138 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:38.140 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:38.140 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:38.140 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:38.141 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:38.141 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:38.141 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:38.142 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:38.142 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:38.138+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:38.241 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:38.240+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:38.283 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:38.283 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:38.283 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:38.283 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:38.292 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:38.341 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:38.341+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:38.394 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:38.394 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:38.394 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:38.394 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:38.394 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:38.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:38.442 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:38.442+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:38.508 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:38.508 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:38.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846442617999998, 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:10:38.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846423422000001, 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:10:38.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846405060999999, 2024-09-18T01:10:38.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846386957, 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:10:38.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:10:38.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.84636836, 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:10:38.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:10:38.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:10:38.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846345360000001, 2024-09-18T01:10:38.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:38.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:38.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846326687999998, 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:10:38.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:10:38.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.541 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846306685000002, 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:10:38.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846288444999999, 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:10:38.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:10:38.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846270031000003, 2024-09-18T01:10:38.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.846248004000003, 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841996225000003, 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.561 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:38.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:10:38.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841985772999998, 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841974082, 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:38.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841960784999998, 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:10:38.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:10:38.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841944714, 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:10:38.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:10:38.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.583 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841928056, 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:10:38.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841908760999999, 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:10:38.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:38.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841880531000001, 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:38.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.596 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.841859407000001, 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:38.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:38.601 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:38.602 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:38.603 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:38.601+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:38.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:10:38.703 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:38.702+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:38.723 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:38.723 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:38.723 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:38.723 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:38.732 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:38.771 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-18T01:10:38.804 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:38.802+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:38.881 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:38.881 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:38.890 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:38.904 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:38.902+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 161.27119792600001, 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:38.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:38.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:38.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:39.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:39.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240171404000002, 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:39.008 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240161825999998, 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240147800000003, 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:39.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:39.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240119026999999, 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:39.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240091331000002, 2024-09-18T01:10:39.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:39.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:39.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.030 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240056883000001, 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:39.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:39.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240027036999997, 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:39.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.240004845999998, 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:39.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:39.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.23996794, 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:39.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:39.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.239946824999997, 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:39.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:39.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:39.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.239916299999997, 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:39.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:39.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234436045999999, 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:39.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234382187000001, 2024-09-18T01:10:39.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:39.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:39.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234304764999997, 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:39.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:39.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:39.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234274651, 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:39.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:39.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234224732999998, 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:39.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234191146000001, 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.234166612000003, 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.233981329999999, 2024-09-18T01:10:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.105 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.106 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:39.107 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:39.104+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:39.206 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:39.205+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:39.211 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:39.211 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:39.211 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:39.211 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.220 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:39.307 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:39.306+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:39.364 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:39.365 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:39.414 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:39.413+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:39.510 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:39.508+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:39.518 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.917517535999998, 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:39.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:39.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916367520999998, 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:39.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916348210000002, 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:39.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916320464000002, 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:39.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916300612000001, 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:39.542 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916280747999998, 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:39.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916262541999998, 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:39.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:39.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916242494999999, 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:39.551 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.555 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916224286999999, 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:39.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:39.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:39.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916206174999999, 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:39.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916188257000002, 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.916170237000003, 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:39.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915617281000003, 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.574 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:39.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.578 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915598557999999, 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:39.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915568774999997, 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:39.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:39.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915504687000002, 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:39.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:39.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915320385999998, 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:39.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:39.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:39.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915302232000002, 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:39.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:39.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915273859999999, 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:39.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:39.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.915251933999997, 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.608 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:39.612 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.614 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:10:39.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:39.615 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:39.611+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:39.714 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:39.713+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:39.725 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:39.726 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:39.726 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:39.726 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.735 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:39.787 INFO:teuthology.orchestra.run.smithi120.stdout:5291 trunc from 0x611e44f to 0xbbab36e 2024-09-18T01:10:39.816 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:39.814+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:39.816 INFO:teuthology.orchestra.run.smithi120.stdout:5292 trunc from 0xbbab36e to 0x8f08e3 2024-09-18T01:10:39.833 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:39.833 INFO:teuthology.orchestra.run.smithi120.stdout:5294 read 0x87822c thru 0x880f26 (0x8cfb bytes) 2024-09-18T01:10:39.833 INFO:teuthology.orchestra.run.smithi120.stdout:5295 punch from 0x5e5cef to 0x5e67e5, (0xaf6 bytes) 2024-09-18T01:10:39.833 INFO:teuthology.orchestra.run.smithi120.stdout:5296 read 0x776c86 thru 0x785602 (0xe97d bytes) 2024-09-18T01:10:39.849 INFO:teuthology.orchestra.run.smithi120.stdout:5297 writesame 0x8a09cea thru 0x8a13a87 data_size 0x4b(0x9d9e bytes) 2024-09-18T01:10:39.855 INFO:teuthology.orchestra.run.smithi120.stdout:5299 trunc from 0x8a13a88 to 0x3c9c3e8 2024-09-18T01:10:39.864 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:39.874 INFO:teuthology.orchestra.run.smithi120.stdout:5302 trunc from 0x3c9c3e8 to 0xc8fe609 2024-09-18T01:10:39.884 INFO:teuthology.orchestra.run.smithi120.stdout:5303 punch from 0x7f93915 to 0x7f9ef48, (0xb633 bytes) 2024-09-18T01:10:39.884 INFO:teuthology.orchestra.run.smithi120.stdout:5304 write 0x36e889d thru 0x36f7e03 (0xf567 bytes) 2024-09-18T01:10:39.887 INFO:teuthology.orchestra.run.smithi120.stdout:5306 read 0x2f6c974 thru 0x2f71671 (0x4cfe bytes) 2024-09-18T01:10:39.888 INFO:teuthology.orchestra.run.smithi120.stdout:5307 trunc from 0xc8fe609 to 0xd52e3b5 2024-09-18T01:10:39.892 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:39.892 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:39.892 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:39.892 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:39.892 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:39.895 INFO:teuthology.orchestra.run.smithi120.stdout:5308 read 0x94657c4 thru 0x9468add (0x331a bytes) 2024-09-18T01:10:39.896 INFO:teuthology.orchestra.run.smithi120.stdout:5310 punch from 0x9256264 to 0x9259092, (0x2e2e bytes) 2024-09-18T01:10:39.896 INFO:teuthology.orchestra.run.smithi120.stdout:5311 write 0x74260e5 thru 0x742620f (0x12b bytes) 2024-09-18T01:10:39.902 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:39.902 INFO:teuthology.orchestra.run.smithi120.stdout:5312 writesame 0xa2e9eaf thru 0xa2ee11c data_size 0xb(0x426e bytes) 2024-09-18T01:10:39.912 INFO:teuthology.orchestra.run.smithi120.stdout:5313 trunc from 0xd52e3b5 to 0x81ae5d9 2024-09-18T01:10:39.916 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:39.914+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:39.921 INFO:teuthology.orchestra.run.smithi120.stdout:5314 writesame 0x762f29b thru 0x76301b9 data_size 0x7(0xf1f bytes) 2024-09-18T01:10:39.922 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:39.922 INFO:teuthology.orchestra.run.smithi120.stdout:5316 read 0x5a0d352 thru 0x5a161e7 (0x8e96 bytes) 2024-09-18T01:10:39.930 INFO:teuthology.orchestra.run.smithi120.stdout:5317 write 0x8c8f083 thru 0x8c9872d (0x96ab bytes) 2024-09-18T01:10:39.937 INFO:teuthology.orchestra.run.smithi120.stdout:5318 trunc from 0x8c9872e to 0x5db9a11 2024-09-18T01:10:39.947 INFO:teuthology.orchestra.run.smithi120.stdout:5319 read 0x287659d thru 0x287d21f (0x6c83 bytes) 2024-09-18T01:10:39.947 INFO:teuthology.orchestra.run.smithi120.stdout:5320 read 0x5e2432 thru 0x5ecc61 (0xa830 bytes) 2024-09-18T01:10:39.948 INFO:teuthology.orchestra.run.smithi120.stdout:5321 trunc from 0x5db9a11 to 0x57c99c5 2024-09-18T01:10:39.953 INFO:teuthology.orchestra.run.smithi120.stdout:5323 read 0x240892 thru 0x242264 (0x19d3 bytes) 2024-09-18T01:10:39.959 INFO:teuthology.orchestra.run.smithi120.stdout:5324 write 0x60d6e2e thru 0x60d7490 (0x663 bytes) 2024-09-18T01:10:39.961 INFO:teuthology.orchestra.run.smithi120.stdout:5325 write 0x2a196ae thru 0x2a24cde (0xb631 bytes) 2024-09-18T01:10:39.967 INFO:teuthology.orchestra.run.smithi120.stdout:5327 trunc from 0x60d7491 to 0x2da9832 2024-09-18T01:10:39.982 INFO:teuthology.orchestra.run.smithi120.stdout:5328 punch from 0x23d7d28 to 0x23e6509, (0xe7e1 bytes) 2024-09-18T01:10:39.982 INFO:teuthology.orchestra.run.smithi120.stdout:5329 read 0x154d48c thru 0x155ce2d (0xf9a2 bytes) 2024-09-18T01:10:39.982 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:39.996 INFO:teuthology.orchestra.run.smithi120.stdout:5331 writesame 0x91ca98b thru 0x91cb05a data_size 0x8(0x6d0 bytes) 2024-09-18T01:10:39.996 INFO:teuthology.orchestra.run.smithi120.stdout:5332 read 0x4105d31 thru 0x4113ed2 (0xe1a2 bytes) 2024-09-18T01:10:39.997 INFO:teuthology.orchestra.run.smithi120.stdout:5333 write 0x875e253 thru 0x876be88 (0xdc36 bytes) 2024-09-18T01:10:40.000 INFO:teuthology.orchestra.run.smithi120.stdout:5334 punch from 0x6b63218 to 0x6b6f7c2, (0xc5aa bytes) 2024-09-18T01:10:40.000 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.000 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.000 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.009 INFO:teuthology.orchestra.run.smithi120.stdout:5339 trunc from 0x91cb05b to 0xfb30fd 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.44048778999999, 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:40.014 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:40.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:40.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.38849664700001, 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:40.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:40.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.38841985100001, 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.027 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:40.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.38391469800001, 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:40.032 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:40.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.38365890200001, 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.040 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:40.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.379633574, 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:40.045 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:40.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:40.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.37949242400001, 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:40.050 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:40.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.055 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.37925952000001, 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.060 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.376640458, 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:40.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:40.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.354301424, 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:40.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:40.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:40.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:40.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40378615, 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.403764409, 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.076 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:40.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:40.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40374400100001, 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:40.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:40.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.084 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.403724342, 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40369100700001, 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:40.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:40.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.093 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40365647500001, 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:40.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40361158499999, 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.098 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.40358521, 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:40.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.403566059, 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:40.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:40.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.111 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.403547491, 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:40.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:40.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:40.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:40.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:40.116 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:40.119 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:40.119 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:40.116+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5340 write 0xe65f33 thru 0xe704dd (0xa5ab bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5341 writesame 0x9fdae2a thru 0x9fe153e data_size 0x1(0x6715 bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5343 write 0x5921498 thru 0x5930e91 (0xf9fa bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5344 write 0xad768d9 thru 0xad78b86 (0x22ae bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5346 punch from 0x4bd3098 to 0x4bd935f, (0x62c7 bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5347 trunc from 0xad78b87 to 0xb34c354 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5348 write 0x93c2725 thru 0x93c6ede (0x47ba bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5349 write 0x8295046 thru 0x82a4ab5 (0xfa70 bytes) 2024-09-18T01:10:40.120 INFO:teuthology.orchestra.run.smithi120.stdout:5350 writesame 0x59e4481 thru 0x59eda2e data_size 0x77(0x95ae bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5352 write 0x58b5930 thru 0x58c2063 (0xc734 bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5353 write 0xb0f5c25 thru 0xb0fd69f (0x7a7b bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5354 writesame 0x17261bc thru 0x1735ca5 data_size 0x1(0xfaea bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5355 read 0x4a71de0 thru 0x4a72f1e (0x113f bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5356 write 0xa1ce2a1 thru 0xa1d3a01 (0x5761 bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5359 punch from 0xa24e7b6 to 0xa25d6ee, (0xef38 bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5360 writesame 0x97829ba thru 0x978714a data_size 0x1f(0x4791 bytes) 2024-09-18T01:10:40.121 INFO:teuthology.orchestra.run.smithi120.stdout:5363 write 0xbc758eb thru 0xbc7fc69 (0xa37f bytes) 2024-09-18T01:10:40.123 INFO:teuthology.orchestra.run.smithi120.stdout:5365 writesame 0x6952130 thru 0x695b42c data_size 0x6f(0x92fd bytes) 2024-09-18T01:10:40.125 INFO:teuthology.orchestra.run.smithi120.stdout:5366 punch from 0x361271d to 0x3612f03, (0x7e6 bytes) 2024-09-18T01:10:40.125 INFO:teuthology.orchestra.run.smithi120.stdout:5367 read 0x73dae7d thru 0x73db210 (0x394 bytes) 2024-09-18T01:10:40.125 INFO:teuthology.orchestra.run.smithi120.stdout:5368 punch from 0xb144cf3 to 0xb1478cf, (0x2bdc bytes) 2024-09-18T01:10:40.125 INFO:teuthology.orchestra.run.smithi120.stdout:5369 read 0x6c04050 thru 0x6c07ed4 (0x3e85 bytes) 2024-09-18T01:10:40.126 INFO:teuthology.orchestra.run.smithi120.stdout:5370 read 0x96119e thru 0x96882f (0x7692 bytes) 2024-09-18T01:10:40.126 INFO:teuthology.orchestra.run.smithi120.stdout:5371 writesame 0x38012fb thru 0x381106f data_size 0x13(0xfd75 bytes) 2024-09-18T01:10:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:5372 punch from 0x458931c to 0x458c9b2, (0x3696 bytes) 2024-09-18T01:10:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:5373 write 0x5bfa94c thru 0x5c06e0f (0xc4c4 bytes) 2024-09-18T01:10:40.132 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.132 INFO:teuthology.orchestra.run.smithi120.stdout:5376 punch from 0xfb0dfd to 0xfb614d, (0x5350 bytes) 2024-09-18T01:10:40.132 INFO:teuthology.orchestra.run.smithi120.stdout:5378 punch from 0x794792 to 0x7a21c2, (0xda30 bytes) 2024-09-18T01:10:40.137 INFO:teuthology.orchestra.run.smithi120.stdout:5379 trunc from 0xbc7fc6a to 0x7e4d2b9 2024-09-18T01:10:40.143 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:40.148 INFO:teuthology.orchestra.run.smithi120.stdout:5380 write 0x6ae1f8d thru 0x6af12c8 (0xf33c bytes) 2024-09-18T01:10:40.149 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.161 INFO:teuthology.orchestra.run.smithi120.stdout:5384 writesame 0xc69e8c6 thru 0xc6ad3d4 data_size 0x5(0xeb0f bytes) 2024-09-18T01:10:40.163 INFO:teuthology.orchestra.run.smithi120.stdout:5385 writesame 0x705145f thru 0x705f017 data_size 0x1(0xdbb9 bytes) 2024-09-18T01:10:40.191 INFO:teuthology.orchestra.run.smithi120.stdout:5386 write 0xa2d7c57 thru 0xa2e2cad (0xb057 bytes) 2024-09-18T01:10:40.193 INFO:teuthology.orchestra.run.smithi120.stdout:5388 read 0xc17766c thru 0xc180f08 (0x989d bytes) 2024-09-18T01:10:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:5389 read 0x1d0af0 thru 0x1df234 (0xe745 bytes) 2024-09-18T01:10:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:5390 punch from 0x32268be to 0x323221d, (0xb95f bytes) 2024-09-18T01:10:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:5393 write 0xae60a62 thru 0xae6616e (0x570d bytes) 2024-09-18T01:10:40.196 INFO:teuthology.orchestra.run.smithi120.stdout:5394 punch from 0xb27ccd1 to 0xb28a083, (0xd3b2 bytes) 2024-09-18T01:10:40.196 INFO:teuthology.orchestra.run.smithi120.stdout:5396 write 0xabc9bec thru 0xabd000a (0x641f bytes) 2024-09-18T01:10:40.198 INFO:teuthology.orchestra.run.smithi120.stdout:5397 write 0x54a09be thru 0x54acc91 (0xc2d4 bytes) 2024-09-18T01:10:40.200 INFO:teuthology.orchestra.run.smithi120.stdout:5398 punch from 0xaad323b to 0xaae21aa, (0xef6f bytes) 2024-09-18T01:10:40.200 INFO:teuthology.orchestra.run.smithi120.stdout:5399 read 0x5fba549 thru 0x5fca07d (0xfb35 bytes) 2024-09-18T01:10:40.201 INFO:teuthology.orchestra.run.smithi120.stdout:5400 punch from 0x635aa9e to 0x6361864, (0x6dc6 bytes) 2024-09-18T01:10:40.207 INFO:teuthology.orchestra.run.smithi120.stdout:5401 writesame 0xce2a199 thru 0xce336f5 data_size 0x1(0x955d bytes) 2024-09-18T01:10:40.209 INFO:teuthology.orchestra.run.smithi120.stdout:5402 read 0x977b27b thru 0x97883a2 (0xd128 bytes) 2024-09-18T01:10:40.210 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.210 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.210 INFO:teuthology.orchestra.run.smithi120.stdout:5406 write 0x5dc2673 thru 0x5dce35f (0xbced bytes) 2024-09-18T01:10:40.214 INFO:teuthology.orchestra.run.smithi120.stdout:5407 trunc from 0xce336f6 to 0xab9d7d3 2024-09-18T01:10:40.218 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:40.217+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:40.227 INFO:teuthology.orchestra.run.smithi120.stdout:5408 punch from 0x7135b23 to 0x7135f6a, (0x447 bytes) 2024-09-18T01:10:40.227 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.228 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.239 INFO:teuthology.orchestra.run.smithi120.stdout:5414 write 0xe2b455f thru 0xe2be61e (0xa0c0 bytes) 2024-09-18T01:10:40.240 INFO:teuthology.orchestra.run.smithi120.stdout:5415 read 0x251aafb thru 0x2525bdc (0xb0e2 bytes) 2024-09-18T01:10:40.241 INFO:teuthology.orchestra.run.smithi120.stdout:5416 writesame 0x5875374 thru 0x587556b data_size 0xe(0x1f8 bytes) 2024-09-18T01:10:40.243 INFO:teuthology.orchestra.run.smithi120.stdout:5418 write 0xb88b032 thru 0xb88d60b (0x25da bytes) 2024-09-18T01:10:40.246 INFO:teuthology.orchestra.run.smithi120.stdout:5421 read 0x4293de3 thru 0x42a3462 (0xf680 bytes) 2024-09-18T01:10:40.246 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.247 INFO:teuthology.orchestra.run.smithi120.stdout:5423 write 0xb2e6d8f thru 0xb2e92ef (0x2561 bytes) 2024-09-18T01:10:40.248 INFO:teuthology.orchestra.run.smithi120.stdout:5425 read 0x37f7633 thru 0x3801f83 (0xa951 bytes) 2024-09-18T01:10:40.249 INFO:teuthology.orchestra.run.smithi120.stdout:5427 write 0x56a8861 thru 0x56afbe2 (0x7382 bytes) 2024-09-18T01:10:40.251 INFO:teuthology.orchestra.run.smithi120.stdout:5428 write 0x1083e2c thru 0x1087bf7 (0x3dcc bytes) 2024-09-18T01:10:40.252 INFO:teuthology.orchestra.run.smithi120.stdout:5431 write 0xba0d2e2 thru 0xba0d85e (0x57d bytes) 2024-09-18T01:10:40.258 INFO:teuthology.orchestra.run.smithi120.stdout:5432 write 0x47db92a thru 0x47eb2e7 (0xf9be bytes) 2024-09-18T01:10:40.260 INFO:teuthology.orchestra.run.smithi120.stdout:5433 punch from 0x768e263 to 0x769a88d, (0xc62a bytes) 2024-09-18T01:10:40.260 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.260 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.260 INFO:teuthology.orchestra.run.smithi120.stdout:5437 writesame 0x534ef79 thru 0x535215c data_size 0x2(0x31e4 bytes) 2024-09-18T01:10:40.265 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:40.265 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:40.265 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:40.266 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:40.266 INFO:teuthology.orchestra.run.smithi120.stdout:5438 punch from 0x3116ff5 to 0x311edfd, (0x7e08 bytes) 2024-09-18T01:10:40.266 INFO:teuthology.orchestra.run.smithi120.stdout:5439 write 0x3344af1 thru 0x33482af (0x37bf bytes) 2024-09-18T01:10:40.268 INFO:teuthology.orchestra.run.smithi120.stdout:5440 trunc from 0xe2be61f to 0xecd917d 2024-09-18T01:10:40.274 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:40.274 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:40.275 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:40.276 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.276 INFO:teuthology.orchestra.run.smithi120.stdout:5443 read 0x5b4112c thru 0x5b4554d (0x4422 bytes) 2024-09-18T01:10:40.276 INFO:teuthology.orchestra.run.smithi120.stdout:5445 writesame 0xc2f5a19 thru 0xc2fb652 data_size 0xf(0x5c3a bytes) 2024-09-18T01:10:40.277 INFO:teuthology.orchestra.run.smithi120.stdout:5446 read 0xa2fdd6c thru 0xa2fe867 (0xafc bytes) 2024-09-18T01:10:40.278 INFO:teuthology.orchestra.run.smithi120.stdout:5447 writesame 0x1fc8fec thru 0x1fcf32a data_size 0x9(0x633f bytes) 2024-09-18T01:10:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:5448 read 0xdc8476a thru 0xdc8514f (0x9e6 bytes) 2024-09-18T01:10:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:5453 read 0x236e8ea thru 0x237dd08 (0xf41f bytes) 2024-09-18T01:10:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.285 INFO:teuthology.orchestra.run.smithi120.stdout:5455 write 0x7ccbbf thru 0x7d811a (0xb55c bytes) 2024-09-18T01:10:40.287 INFO:teuthology.orchestra.run.smithi120.stdout:5457 write 0xce9a15b thru 0xcea71c4 (0xd06a bytes) 2024-09-18T01:10:40.291 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:40.292 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:40.292 INFO:teuthology.orchestra.run.smithi120.stdout:5458 write 0xebd6394 thru 0xebda2ac (0x3f19 bytes) 2024-09-18T01:10:40.298 INFO:teuthology.orchestra.run.smithi120.stdout:5460 write 0xe2f35e1 thru 0xe2fff0f (0xc92f bytes) 2024-09-18T01:10:40.299 INFO:teuthology.orchestra.run.smithi120.stdout:5462 punch from 0xdcb23c3 to 0xdcbc851, (0xa48e bytes) 2024-09-18T01:10:40.299 INFO:teuthology.orchestra.run.smithi120.stdout:5463 writesame 0xec2ef73 thru 0xec31e3b data_size 0x1d(0x2ec9 bytes) 2024-09-18T01:10:40.304 INFO:teuthology.orchestra.run.smithi120.stdout:5464 punch from 0xb193556 to 0xb19747d, (0x3f27 bytes) 2024-09-18T01:10:40.304 INFO:teuthology.orchestra.run.smithi120.stdout:5465 write 0x1f0e705 thru 0x1f1e02b (0xf927 bytes) 2024-09-18T01:10:40.306 INFO:teuthology.orchestra.run.smithi120.stdout:5467 write 0x2fdbdd0 thru 0x2feb852 (0xfa83 bytes) 2024-09-18T01:10:40.308 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:40.308 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:40.308 INFO:teuthology.orchestra.run.smithi120.stdout:5468 write 0x7fdcddf thru 0x7fdfdd8 (0x2ffa bytes) 2024-09-18T01:10:40.318 INFO:teuthology.orchestra.run.smithi120.stdout:5470 trunc from 0xecd917d to 0x7b7d192 2024-09-18T01:10:40.326 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:40.326 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:40.334 INFO:teuthology.orchestra.run.smithi120.stdout:5471 punch from 0x3e8f0d5 to 0x3e911d1, (0x20fc bytes) 2024-09-18T01:10:40.334 INFO:teuthology.orchestra.run.smithi120.stdout:5472 read 0x1f52f63 thru 0x1f5f33f (0xc3dd bytes) 2024-09-18T01:10:40.334 INFO:teuthology.orchestra.run.smithi120.stdout:5473 read 0x2ee1866 thru 0x2eecdd1 (0xb56c bytes) 2024-09-18T01:10:40.335 INFO:teuthology.orchestra.run.smithi120.stdout:5474 punch from 0x47f7664 to 0x480213f, (0xaadb bytes) 2024-09-18T01:10:40.335 INFO:teuthology.orchestra.run.smithi120.stdout:5475 punch from 0x50080ba to 0x5011d9a, (0x9ce0 bytes) 2024-09-18T01:10:40.335 INFO:teuthology.orchestra.run.smithi120.stdout:5476 read 0x5d1fc5f thru 0x5d268d5 (0x6c77 bytes) 2024-09-18T01:10:40.343 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:40.344 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:40.345 INFO:teuthology.orchestra.run.smithi120.stdout:5477 writesame 0x95a24f7 thru 0x95b228c data_size 0x7(0xfd96 bytes) 2024-09-18T01:10:40.346 INFO:teuthology.orchestra.run.smithi120.stdout:5478 read 0x34eac0e thru 0x34f32f1 (0x86e4 bytes) 2024-09-18T01:10:40.354 INFO:teuthology.orchestra.run.smithi120.stdout:5479 trunc from 0x95b228d to 0x3156c63 2024-09-18T01:10:40.359 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:40.358+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:40.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:40.362 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:40.376 INFO:teuthology.orchestra.run.smithi120.stdout:5481 read 0x2c575f1 thru 0x2c6494a (0xd35a bytes) 2024-09-18T01:10:40.379 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:40.382 INFO:teuthology.orchestra.run.smithi120.stdout:5482 writesame 0x383e2fe thru 0x3842a09 data_size 0x2(0x470c bytes) 2024-09-18T01:10:40.393 INFO:teuthology.orchestra.run.smithi120.stdout:5484 write 0x81b8963 thru 0x81c0ee5 (0x8583 bytes) 2024-09-18T01:10:40.397 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:40.397 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:40.404 INFO:teuthology.orchestra.run.smithi120.stdout:5485 writesame 0xb7fa1bd thru 0xb7fa947 data_size 0x1(0x78b bytes) 2024-09-18T01:10:40.404 INFO:teuthology.orchestra.run.smithi120.stdout:5486 read 0x7858b67 thru 0x785be40 (0x32da bytes) 2024-09-18T01:10:40.404 INFO:teuthology.orchestra.run.smithi120.stdout:5489 write 0xb599b8f thru 0xb5a99d2 (0xfe44 bytes) 2024-09-18T01:10:40.413 INFO:teuthology.orchestra.run.smithi120.stdout:5490 write 0xbd9aac4 thru 0xbd9f5c0 (0x4afd bytes) 2024-09-18T01:10:40.417 INFO:teuthology.orchestra.run.smithi120.stdout:5491 read 0x2799dbc thru 0x279de94 (0x40d9 bytes) 2024-09-18T01:10:40.420 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:40.419+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:40.420 INFO:teuthology.orchestra.run.smithi120.stdout:5492 trunc from 0xbd9f5c1 to 0x94bfae4 2024-09-18T01:10:40.427 INFO:teuthology.orchestra.run.smithi120.stdout:5493 punch from 0x3665145 to 0x36699e3, (0x489e bytes) 2024-09-18T01:10:40.428 INFO:teuthology.orchestra.run.smithi120.stdout:5495 read 0x595ac11 thru 0x596abb0 (0xffa0 bytes) 2024-09-18T01:10:40.428 INFO:teuthology.orchestra.run.smithi120.stdout:5496 read 0xc2f15f thru 0xc30a70 (0x1912 bytes) 2024-09-18T01:10:40.428 INFO:teuthology.orchestra.run.smithi120.stdout:5497 read 0x60f83d9 thru 0x6100d8d (0x89b5 bytes) 2024-09-18T01:10:40.428 INFO:teuthology.orchestra.run.smithi120.stdout:5499 read 0x33b3b4a thru 0x33c24d2 (0xe989 bytes) 2024-09-18T01:10:40.429 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.429 INFO:teuthology.orchestra.run.smithi120.stdout:5501 write 0x569d091 thru 0x56ac4e8 (0xf458 bytes) 2024-09-18T01:10:40.429 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:40.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:40.430 INFO:teuthology.orchestra.run.smithi120.stdout:5502 read 0x3784452 thru 0x378dac7 (0x9676 bytes) 2024-09-18T01:10:40.430 INFO:teuthology.orchestra.run.smithi120.stdout:5503 writesame 0x411e5c5 thru 0x4122b53 data_size 0x1(0x458f bytes) 2024-09-18T01:10:40.439 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:40.440 INFO:teuthology.orchestra.run.smithi120.stdout:5504 punch from 0x1c9f38a to 0x1caccbf, (0xd935 bytes) 2024-09-18T01:10:40.440 INFO:teuthology.orchestra.run.smithi120.stdout:5505 writesame 0x497377c thru 0x4979b0b data_size 0x3b(0x6390 bytes) 2024-09-18T01:10:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.450 INFO:teuthology.orchestra.run.smithi120.stdout:5507 trunc from 0x94bfae4 to 0x4d3d5ad 2024-09-18T01:10:40.473 INFO:teuthology.orchestra.run.smithi120.stdout:5508 trunc from 0x4d3d5ad to 0xa3159f 2024-09-18T01:10:40.498 INFO:teuthology.orchestra.run.smithi120.stdout:5509 punch from 0x73d33e to 0x7418ad, (0x456f bytes) 2024-09-18T01:10:40.499 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.499 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.499 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.504 INFO:teuthology.orchestra.run.smithi120.stdout:5514 writesame 0xda76ee thru 0xdb2346 data_size 0x4d(0xac59 bytes) 2024-09-18T01:10:40.521 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:40.519+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:40.525 INFO:teuthology.orchestra.run.smithi120.stdout:5515 writesame 0xd4cf387 thru 0xd4d9d93 data_size 0x15(0xaa0d bytes) 2024-09-18T01:10:40.525 INFO:teuthology.orchestra.run.smithi120.stdout:5516 read 0x26cb960 thru 0x26d27a3 (0x6e44 bytes) 2024-09-18T01:10:40.525 INFO:teuthology.orchestra.run.smithi120.stdout:5517 write 0xb7bbd44 thru 0xb7c70bf (0xb37c bytes) 2024-09-18T01:10:40.529 INFO:teuthology.orchestra.run.smithi120.stdout:5519 read 0x5f85a63 thru 0x5f87c53 (0x21f1 bytes) 2024-09-18T01:10:40.529 INFO:teuthology.orchestra.run.smithi120.stdout:5520 writesame 0xe1888 thru 0xe4700 data_size 0x1(0x2e79 bytes) 2024-09-18T01:10:40.535 INFO:teuthology.orchestra.run.smithi120.stdout:5522 read 0x9274e5f thru 0x9277cd6 (0x2e78 bytes) 2024-09-18T01:10:40.535 INFO:teuthology.orchestra.run.smithi120.stdout:5523 read 0x14b7da2 thru 0x14bab2b (0x2d8a bytes) 2024-09-18T01:10:40.535 INFO:teuthology.orchestra.run.smithi120.stdout:5524 read 0x60851f8 thru 0x608e368 (0x9171 bytes) 2024-09-18T01:10:40.543 INFO:teuthology.orchestra.run.smithi120.stdout:5525 writesame 0xeb25451 thru 0xeb26cba data_size 0xa(0x186a bytes) 2024-09-18T01:10:40.543 INFO:teuthology.orchestra.run.smithi120.stdout:5526 read 0xc481755 thru 0xc48ea44 (0xd2f0 bytes) 2024-09-18T01:10:40.543 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.544 INFO:teuthology.orchestra.run.smithi120.stdout:5528 writesame 0xde3a47b thru 0xde43a21 data_size 0xd(0x95a7 bytes) 2024-09-18T01:10:40.550 INFO:teuthology.orchestra.run.smithi120.stdout:5529 write 0x2671383 thru 0x26753f2 (0x4070 bytes) 2024-09-18T01:10:40.554 INFO:teuthology.orchestra.run.smithi120.stdout:5530 write 0x5f1a9ec thru 0x5f215c7 (0x6bdc bytes) 2024-09-18T01:10:40.557 INFO:teuthology.orchestra.run.smithi120.stdout:5531 read 0xcfed99f thru 0xcff824a (0xa8ac bytes) 2024-09-18T01:10:40.557 INFO:teuthology.orchestra.run.smithi120.stdout:5533 write 0x876840c thru 0x876a871 (0x2466 bytes) 2024-09-18T01:10:40.559 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.559 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.559 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.559 INFO:teuthology.orchestra.run.smithi120.stdout:5537 write 0x7179b86 thru 0x717a7cd (0xc48 bytes) 2024-09-18T01:10:40.563 INFO:teuthology.orchestra.run.smithi120.stdout:5538 read 0x9368ac thru 0x93b75a (0x4eaf bytes) 2024-09-18T01:10:40.564 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.564 INFO:teuthology.orchestra.run.smithi120.stdout:5540 punch from 0x1938145 to 0x193aa84, (0x293f bytes) 2024-09-18T01:10:40.564 INFO:teuthology.orchestra.run.smithi120.stdout:5541 writesame 0x1c492d4 thru 0x1c4962f data_size 0x2(0x35c bytes) 2024-09-18T01:10:40.567 INFO:teuthology.orchestra.run.smithi120.stdout:5542 write 0xb58ac1c thru 0xb59337b (0x8760 bytes) 2024-09-18T01:10:40.568 INFO:teuthology.orchestra.run.smithi120.stdout:5543 writesame 0x4bdf7c3 thru 0x4be2190 data_size 0x1(0x29ce bytes) 2024-09-18T01:10:40.587 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:40.588 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:40.589 INFO:teuthology.orchestra.run.smithi120.stdout:5547 trunc from 0xeb26cbb to 0x45b903e 2024-09-18T01:10:40.598 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:40.605 INFO:teuthology.orchestra.run.smithi120.stdout:5549 read 0x392edaf thru 0x3932b4a (0x3d9c bytes) 2024-09-18T01:10:40.612 INFO:teuthology.orchestra.run.smithi120.stdout:5551 write 0x5a351b3 thru 0x5a38dd2 (0x3c20 bytes) 2024-09-18T01:10:40.615 INFO:teuthology.orchestra.run.smithi120.stdout:5554 write 0x48e7aaf thru 0x48e87c5 (0xd17 bytes) 2024-09-18T01:10:40.619 INFO:teuthology.orchestra.run.smithi120.stdout:5555 read 0x3839d7f thru 0x383ee33 (0x50b5 bytes) 2024-09-18T01:10:40.619 INFO:teuthology.orchestra.run.smithi120.stdout:5556 read 0x555957 thru 0x561e91 (0xc53b bytes) 2024-09-18T01:10:40.619 INFO:teuthology.orchestra.run.smithi120.stdout:5560 write 0x18d87ca thru 0x18e5fe9 (0xd820 bytes) 2024-09-18T01:10:40.621 INFO:teuthology.orchestra.run.smithi120.stdout:5562 write 0x41bbbf7 thru 0x41cb898 (0xfca2 bytes) 2024-09-18T01:10:40.621 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:40.620+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:40.624 INFO:teuthology.orchestra.run.smithi120.stdout:5563 read 0x200d6d0 thru 0x201b218 (0xdb49 bytes) 2024-09-18T01:10:40.624 INFO:teuthology.orchestra.run.smithi120.stdout:5565 writesame 0x4b68286 thru 0x4b75b05 data_size 0x1(0xd880 bytes) 2024-09-18T01:10:40.647 INFO:teuthology.orchestra.run.smithi120.stdout:5566 read 0x2fa9946 thru 0x2fb5119 (0xb7d4 bytes) 2024-09-18T01:10:40.647 INFO:teuthology.orchestra.run.smithi120.stdout:5567 read 0x38c5672 thru 0x38c870a (0x3099 bytes) 2024-09-18T01:10:40.647 INFO:teuthology.orchestra.run.smithi120.stdout:5568 trunc from 0x5a38dd3 to 0x5e8598c 2024-09-18T01:10:40.652 INFO:teuthology.orchestra.run.smithi120.stdout:5570 writesame 0x5ab7f48 thru 0x5ab87fb data_size 0x2(0x8b4 bytes) 2024-09-18T01:10:40.663 INFO:teuthology.orchestra.run.smithi120.stdout:5571 trunc from 0x5e8598c to 0xce806c9 2024-09-18T01:10:40.670 INFO:teuthology.orchestra.run.smithi120.stdout:5572 write 0x6602f93 thru 0x660d6d3 (0xa741 bytes) 2024-09-18T01:10:40.681 INFO:teuthology.orchestra.run.smithi120.stdout:5573 write 0xe54907f thru 0xe54d8d9 (0x485b bytes) 2024-09-18T01:10:40.683 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.683 INFO:teuthology.orchestra.run.smithi120.stdout:5575 punch from 0x22a30d7 to 0x22ae2dc, (0xb205 bytes) 2024-09-18T01:10:40.683 INFO:teuthology.orchestra.run.smithi120.stdout:5576 writesame 0x9ddd4ae thru 0x9ddf3e9 data_size 0x2(0x1f3c bytes) 2024-09-18T01:10:40.709 INFO:teuthology.orchestra.run.smithi120.stdout:5578 trunc from 0xe54d8da to 0x88c78f 2024-09-18T01:10:40.713 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:40.713 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:40.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:40.724 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:40.721+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:40.731 INFO:teuthology.orchestra.run.smithi120.stdout:5579 trunc from 0x88c78f to 0x20e200c 2024-09-18T01:10:40.744 INFO:teuthology.orchestra.run.smithi120.stdout:5580 writesame 0x39a482c thru 0x39add7f data_size 0x26(0x9554 bytes) 2024-09-18T01:10:40.744 INFO:teuthology.orchestra.run.smithi120.stdout:5581 punch from 0xf005a9 to 0xf0ca52, (0xc4a9 bytes) 2024-09-18T01:10:40.745 INFO:teuthology.orchestra.run.smithi120.stdout:5582 read 0x2840e16 thru 0x284b37a (0xa565 bytes) 2024-09-18T01:10:40.745 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.752 INFO:teuthology.orchestra.run.smithi120.stdout:5584 writesame 0x5a0eb56 thru 0x5a105a0 data_size 0x1(0x1a4b bytes) 2024-09-18T01:10:40.765 INFO:teuthology.orchestra.run.smithi120.stdout:5585 writesame 0x97c8a77 thru 0x97d385a data_size 0x3e(0xade4 bytes) 2024-09-18T01:10:40.766 INFO:teuthology.orchestra.run.smithi120.stdout:5587 punch from 0x209010c to 0x209e4c8, (0xe3bc bytes) 2024-09-18T01:10:40.768 INFO:teuthology.orchestra.run.smithi120.stdout:5588 trunc from 0x97d385b to 0xbc94e86 2024-09-18T01:10:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:5589 read 0x1a14a21 thru 0x1a17158 (0x2738 bytes) 2024-09-18T01:10:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:5592 punch from 0x2820ff1 to 0x282f966, (0xe975 bytes) 2024-09-18T01:10:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:5593 read 0x42949b3 thru 0x429a909 (0x5f57 bytes) 2024-09-18T01:10:40.775 INFO:teuthology.orchestra.run.smithi120.stdout:5594 punch from 0x5965320 to 0x5966671, (0x1351 bytes) 2024-09-18T01:10:40.780 INFO:teuthology.orchestra.run.smithi120.stdout:5595 trunc from 0xbc94e86 to 0x7ccd3d5 2024-09-18T01:10:40.786 INFO:teuthology.orchestra.run.smithi120.stdout:5596 writesame 0x1f0dfe4 thru 0x1f1113e data_size 0x5f(0x315b bytes) 2024-09-18T01:10:40.790 INFO:teuthology.orchestra.run.smithi120.stdout:5597 punch from 0x348845b to 0x34933e4, (0xaf89 bytes) 2024-09-18T01:10:40.790 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.793 INFO:teuthology.orchestra.run.smithi120.stdout:5599 trunc from 0x7ccd3d5 to 0x535e0ef 2024-09-18T01:10:40.819 INFO:teuthology.orchestra.run.smithi120.stdout:5601 write 0xe3da922 thru 0xe3e7fc6 (0xd6a5 bytes) 2024-09-18T01:10:40.822 INFO:teuthology.orchestra.run.smithi120.stdout:5602 write 0xcd223dc thru 0xcd2aa46 (0x866b bytes) 2024-09-18T01:10:40.824 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.824 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:40.824 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:40.822+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:40.825 INFO:teuthology.orchestra.run.smithi120.stdout:5606 trunc from 0xe3e7fc7 to 0xce668ce 2024-09-18T01:10:40.840 INFO:teuthology.orchestra.run.smithi120.stdout:5607 write 0xde10f2e thru 0xde20056 (0xf129 bytes) 2024-09-18T01:10:40.846 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:40.846 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:40.848 INFO:teuthology.orchestra.run.smithi120.stdout:5610 writesame 0xe72c14a thru 0xe733176 data_size 0xd(0x702d bytes) 2024-09-18T01:10:40.857 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:40.862 INFO:teuthology.orchestra.run.smithi120.stdout:5611 trunc from 0xe733177 to 0x4445207 2024-09-18T01:10:40.871 INFO:teuthology.orchestra.run.smithi120.stdout:5612 write 0x3ff6d3a thru 0x3ffb3d4 (0x469b bytes) 2024-09-18T01:10:40.875 INFO:teuthology.orchestra.run.smithi120.stdout:5613 read 0x10d948e thru 0x10df691 (0x6204 bytes) 2024-09-18T01:10:40.875 INFO:teuthology.orchestra.run.smithi120.stdout:5614 read 0x2c895bb thru 0x2c93319 (0x9d5f bytes) 2024-09-18T01:10:40.885 INFO:teuthology.orchestra.run.smithi120.stdout:5616 write 0x872fac2 thru 0x8734722 (0x4c61 bytes) 2024-09-18T01:10:40.896 INFO:teuthology.orchestra.run.smithi120.stdout:5617 writesame 0xbb95cd6 thru 0xbb9e88b data_size 0x1(0x8bb6 bytes) 2024-09-18T01:10:40.896 INFO:teuthology.orchestra.run.smithi120.stdout:5620 read 0x2385def thru 0x2389ef9 (0x410b bytes) 2024-09-18T01:10:40.896 INFO:teuthology.orchestra.run.smithi120.stdout:5621 read 0x8c8b9a7 thru 0x8c925ee (0x6c48 bytes) 2024-09-18T01:10:40.899 INFO:teuthology.orchestra.run.smithi120.stdout:5622 trunc from 0xbb9e88c to 0x932b244 2024-09-18T01:10:40.906 INFO:teuthology.orchestra.run.smithi120.stdout:5623 punch from 0x921c3bd to 0x92293fe, (0xd041 bytes) 2024-09-18T01:10:40.906 INFO:teuthology.orchestra.run.smithi120.stdout:5625 write 0x684e1c6 thru 0x685cf90 (0xedcb bytes) 2024-09-18T01:10:40.908 INFO:teuthology.orchestra.run.smithi120.stdout:5629 punch from 0x47baca7 to 0x47bb186, (0x4df bytes) 2024-09-18T01:10:40.919 INFO:teuthology.orchestra.run.smithi120.stdout:5630 trunc from 0x932b244 to 0x652563 2024-09-18T01:10:40.925 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:40.924+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:40.944 INFO:teuthology.orchestra.run.smithi120.stdout:5631 read 0x996bc thru 0x9bf14 (0x2859 bytes) 2024-09-18T01:10:40.962 INFO:teuthology.orchestra.run.smithi120.stdout:5632 write 0x88d7fdb thru 0x88e18cd (0x98f3 bytes) 2024-09-18T01:10:40.966 INFO:teuthology.orchestra.run.smithi120.stdout:5633 read 0x6ca4bb9 thru 0x6cacbfc (0x8044 bytes) 2024-09-18T01:10:40.975 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:40.975 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:40.978 INFO:teuthology.orchestra.run.smithi120.stdout:5634 write 0xd8f1282 thru 0xd8f58bb (0x463a bytes) 2024-09-18T01:10:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:5635 read 0x1a2f26e thru 0x1a32a5e (0x37f1 bytes) 2024-09-18T01:10:40.981 INFO:teuthology.orchestra.run.smithi120.stdout:5636 read 0x8f66e66 thru 0x8f6c27d (0x5418 bytes) 2024-09-18T01:10:40.982 INFO:teuthology.orchestra.run.smithi120.stdout:5637 writesame 0xd34d6f4 thru 0xd34d896 data_size 0x1(0x1a3 bytes) 2024-09-18T01:10:40.985 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:40.985 INFO:teuthology.orchestra.run.smithi120.stdout:5638 punch from 0xb13b102 to 0xb1434cf, (0x83cd bytes) 2024-09-18T01:10:40.985 INFO:teuthology.orchestra.run.smithi120.stdout:5639 write 0x8fbeda9 thru 0x8fcb869 (0xcac1 bytes) 2024-09-18T01:10:40.987 INFO:teuthology.orchestra.run.smithi120.stdout:5640 write 0x9227f0c thru 0x922d01e (0x5113 bytes) 2024-09-18T01:10:40.996 INFO:teuthology.orchestra.run.smithi120.stdout:5641 trunc from 0xd8f58bc to 0x89b632d 2024-09-18T01:10:41.007 INFO:teuthology.orchestra.run.smithi120.stdout:5642 read 0x88995f5 thru 0x88a87a8 (0xf1b4 bytes) 2024-09-18T01:10:41.007 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.017 INFO:teuthology.orchestra.run.smithi120.stdout:5644 writesame 0xcbf519d thru 0xcbfec1e data_size 0x1(0x9a82 bytes) 2024-09-18T01:10:41.017 INFO:teuthology.orchestra.run.smithi120.stdout:5645 writesame 0x83263bc thru 0x8331d6f data_size 0xa(0xb9b4 bytes) 2024-09-18T01:10:41.022 INFO:teuthology.orchestra.run.smithi120.stdout:5648 read 0x7d71bd5 thru 0x7d77b0e (0x5f3a bytes) 2024-09-18T01:10:41.022 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.026 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:41.025+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:41.031 INFO:teuthology.orchestra.run.smithi120.stdout:5650 write 0xe1fdacc thru 0xe201f2e (0x4463 bytes) 2024-09-18T01:10:41.032 INFO:teuthology.orchestra.run.smithi120.stdout:5651 punch from 0x5cac519 to 0x5cb8f37, (0xca1e bytes) 2024-09-18T01:10:41.033 INFO:teuthology.orchestra.run.smithi120.stdout:5652 read 0x56c53 thru 0x6298a (0xbd38 bytes) 2024-09-18T01:10:41.035 INFO:teuthology.orchestra.run.smithi120.stdout:5654 punch from 0x5ff1f9f to 0x5ff373f, (0x17a0 bytes) 2024-09-18T01:10:41.040 INFO:teuthology.orchestra.run.smithi120.stdout:5655 write 0xea84271 thru 0xea890d0 (0x4e60 bytes) 2024-09-18T01:10:41.042 INFO:teuthology.orchestra.run.smithi120.stdout:5656 write 0x7b847e4 thru 0x7b869fc (0x2219 bytes) 2024-09-18T01:10:41.045 INFO:teuthology.orchestra.run.smithi120.stdout:5657 trunc from 0xea890d1 to 0xe9f93fd 2024-09-18T01:10:41.052 INFO:teuthology.orchestra.run.smithi120.stdout:5659 read 0x49e8dd2 thru 0x49f1793 (0x89c2 bytes) 2024-09-18T01:10:41.052 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.053 INFO:teuthology.orchestra.run.smithi120.stdout:5662 read 0x438d99e thru 0x4395ba0 (0x8203 bytes) 2024-09-18T01:10:41.053 INFO:teuthology.orchestra.run.smithi120.stdout:5663 writesame 0x93662d5 thru 0x93755e5 data_size 0x19(0xf311 bytes) 2024-09-18T01:10:41.055 INFO:teuthology.orchestra.run.smithi120.stdout:5664 write 0x72dd3f0 thru 0x72e3507 (0x6118 bytes) 2024-09-18T01:10:41.072 INFO:teuthology.orchestra.run.smithi120.stdout:5665 trunc from 0xe9f93fd to 0x1e06ca9 2024-09-18T01:10:41.090 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.102 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.102 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.108 INFO:teuthology.orchestra.run.smithi120.stdout:5667 write 0xa0594e4 thru 0xa063400 (0x9f1d bytes) 2024-09-18T01:10:41.112 INFO:teuthology.orchestra.run.smithi120.stdout:5668 trunc from 0xa063401 to 0x964365a 2024-09-18T01:10:41.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:41.118 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.119 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.129 INFO:teuthology.orchestra.run.smithi120.stdout:5671 trunc from 0x964365a to 0x178fce 2024-09-18T01:10:41.130 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:41.128+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:41.165 INFO:teuthology.orchestra.run.smithi120.stdout:5672 write 0xa4644c5 thru 0xa471815 (0xd351 bytes) 2024-09-18T01:10:41.179 INFO:teuthology.orchestra.run.smithi120.stdout:5673 writesame 0xe866435 thru 0xe874327 data_size 0x19(0xdef3 bytes) 2024-09-18T01:10:41.179 INFO:teuthology.orchestra.run.smithi120.stdout:5674 write 0x6e3bcfa thru 0x6e4a790 (0xea97 bytes) 2024-09-18T01:10:41.182 INFO:teuthology.orchestra.run.smithi120.stdout:5675 write 0xde148f9 thru 0xde1cc5e (0x8366 bytes) 2024-09-18T01:10:41.187 INFO:teuthology.orchestra.run.smithi120.stdout:5676 write 0xd672b0b thru 0xd67653b (0x3a31 bytes) 2024-09-18T01:10:41.190 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.190 INFO:teuthology.orchestra.run.smithi120.stdout:5679 writesame 0xcc657bf thru 0xcc66fd4 data_size 0x1(0x1816 bytes) 2024-09-18T01:10:41.194 INFO:teuthology.orchestra.run.smithi120.stdout:5680 write 0x78b16bc thru 0x78bcc6c (0xb5b1 bytes) 2024-09-18T01:10:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:5681 punch from 0x9e2a76 to 0x9eb943, (0x8ecd bytes) 2024-09-18T01:10:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:5683 read 0xc297e68 thru 0xc2a501e (0xd1b7 bytes) 2024-09-18T01:10:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.196 INFO:teuthology.orchestra.run.smithi120.stdout:5686 write 0x39d867f thru 0x39dc197 (0x3b19 bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5688 read 0x2e8bd12 thru 0x2e8f23b (0x352a bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5689 punch from 0x1185ee7 to 0x1188fab, (0x30c4 bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5694 read 0x32abd1a thru 0x32af7b4 (0x3a9b bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5696 read 0x8692a76 thru 0x8695b79 (0x3104 bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5698 read 0x2e245a thru 0x2f0b96 (0xe73d bytes) 2024-09-18T01:10:41.200 INFO:teuthology.orchestra.run.smithi120.stdout:5699 write 0xaec8b3b thru 0xaed8235 (0xf6fb bytes) 2024-09-18T01:10:41.203 INFO:teuthology.orchestra.run.smithi120.stdout:5700 writesame 0xa9279b7 thru 0xa935c75 data_size 0xb(0xe2bf bytes) 2024-09-18T01:10:41.206 INFO:teuthology.orchestra.run.smithi120.stdout:5703 read 0xd69df03 thru 0xd6ab70d (0xd80b bytes) 2024-09-18T01:10:41.207 INFO:teuthology.orchestra.run.smithi120.stdout:5705 writesame 0x48c5dfb thru 0x48d18ce data_size 0x16(0xbad4 bytes) 2024-09-18T01:10:41.209 INFO:teuthology.orchestra.run.smithi120.stdout:5706 read 0x4e91000 thru 0x4e9148e (0x48f bytes) 2024-09-18T01:10:41.209 INFO:teuthology.orchestra.run.smithi120.stdout:5711 punch from 0xe5c20e2 to 0xe5c8712, (0x6630 bytes) 2024-09-18T01:10:41.209 INFO:teuthology.orchestra.run.smithi120.stdout:5712 punch from 0x58bd33 to 0x58e055, (0x2322 bytes) 2024-09-18T01:10:41.209 INFO:teuthology.orchestra.run.smithi120.stdout:5713 write 0x73f291b thru 0x7401ad6 (0xf1bc bytes) 2024-09-18T01:10:41.212 INFO:teuthology.orchestra.run.smithi120.stdout:5714 punch from 0xdabec7 to 0xdb573d, (0x9876 bytes) 2024-09-18T01:10:41.212 INFO:teuthology.orchestra.run.smithi120.stdout:5716 clone 45 order 21 su 131072 sc 10 2024-09-18T01:10:41.228 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:41.227+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:41.230 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.230 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.239 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:41.328 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:41.327+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:41.381 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.381 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:41.431 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:41.429+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:41.530 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:41.530+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:41.532 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.533 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.541 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:41.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:10:41.645 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.645 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.646 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:41.644+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:41.655 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:41.732 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:41.731+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:41.806 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.806 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.815 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:41.833 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:41.832+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:41.919 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:41.920 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:41.928 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:41.974 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:41.972+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:42.032 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:42.032 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:42.034 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:42.034+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:42.041 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:42.135 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:42.134+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:42.184 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:42.185 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:42.193 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:42.237 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:42.236+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:42.304 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:42.305 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:42.313 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:42.337 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:42.335+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:42.410 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:42.454 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:42.452+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:42.457 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:42.457 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:42.466 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:42.530 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:42.530 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:42.538 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:42.536+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:42.546 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:42.546 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:42.562 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:42.562 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:42.570 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:42.570 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:42.577 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:42.578 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:42.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:42.594 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:42.594 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:42.610 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:42.610 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:42.626 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:42.627 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:42.639 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:42.637+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:42.643 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:42.643 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:42.684 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:42.684 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:42.684 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:42.684 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:42.684 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-18T01:10:42.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-18T01:10:42.692 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:10:42.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:42.750 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:42.748+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:42.840 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:42.838+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:42.853 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:42.854 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:42.854 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:42.854 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:42.854 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:42.863 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:42.941 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:42.939+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:43.008 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.407452894000002, 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:10:43.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:10:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406302879000002, 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406283567999999, 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406255821999999, 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:10:43.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:10:43.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.028 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406235969999997, 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:10:43.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406216106000002, 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.033 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:10:43.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.037 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406197900000002, 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406177853000003, 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:10:43.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:10:43.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:43.047 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406159645000002, 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:10:43.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406141533000003, 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:10:43.052 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:43.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:10:43.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406123614999998, 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.057 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:10:43.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.406105595, 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:10:43.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:10:43.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405552639, 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:10:43.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:43.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405533916000003, 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:43.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:10:43.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405504133000001, 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:10:43.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405440044999999, 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:10:43.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405255744000002, 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:10:43.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:10:43.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:10:43.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405237589999999, 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:43.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:10:43.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405209218000003, 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.095 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:10:43.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.405187292000001, 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:10:43.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.104 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.105 INFO:tasks.ceph.mon.a.smithi120.stderr:2024-09-18T01:10:43.029+0000 7f77b87d3640 -1 mon.a@0(leader).osd e207 definitely_dead 0 2024-09-18T01:10:43.106 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:43.106 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:43.103+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:43.207 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:43.205+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:43.209 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.209 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:43.209 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:43.209 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:43.307 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:43.305+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:43.362 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.362 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:43.362 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:43.362 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:43.362 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:43.412 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:43.410+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:43.508 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:43.506+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:51933 3.10 3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:43.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.94339558199999, 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98646169699999997, 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 51933 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571062+0000", 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571065+0000", 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6220000000000001e-06 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571576+0000", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051131899999999999 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571593+0000", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6704000000000001e-05 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571600+0000", 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:10:43.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571671+0000", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1035e-05 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.571684+0000", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2899e-05 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554622+0000", 2024-09-18T01:10:43.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98293849899999997 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554748+0000", 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125897 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.554852+0000", 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010346 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556533+0000", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016809069999999999 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557441+0000", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000908616 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557495+0000", 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557524+0000", 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8819999999999999e-05 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52375 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.89140443900001, 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93315232199999998, 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52375 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623053+0000", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623055+0000", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623060+0000", 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2800000000000003e-06 2024-09-18T01:10:43.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623074+0000", 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4052e-05 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623082+0000", 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623174+0000", 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1701999999999997e-05 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623186+0000", 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.164e-05 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556120+0000", 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93293410200000004 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556205+0000", 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5785000000000006e-05 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52379 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.89132764300001, 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93320734599999999, 2024-09-18T01:10:43.528 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52379 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623130+0000", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623131+0000", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7700000000000004e-07 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623134+0000", 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:10:43.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623143+0000", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1549999999999996e-06 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623154+0000", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0889e-05 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623232+0000", 2024-09-18T01:10:43.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7554999999999999e-05 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.623243+0000", 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1452e-05 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556227+0000", 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93298413400000002 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556337+0000", 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011001399999999999 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52391 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.88682248999999, 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92879877499999997, 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52391 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627635+0000", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627636+0000", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627644+0000", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1790000000000008e-06 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627659+0000", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.482e-05 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627667+0000", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4680000000000003e-06 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627725+0000", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8810000000000001e-05 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627735+0000", 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556364+0000", 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92862919099999996 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556434+0000", 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999995e-05 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52395 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.88656669400001, 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92864427000000005, 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.538 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52395 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627891+0000", 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627892+0000", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6599999999999995e-07 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627901+0000", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5659999999999996e-06 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627911+0000", 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.713e-06 2024-09-18T01:10:43.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627918+0000", 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627961+0000", 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3637999999999998e-05 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.627968+0000", 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3010000000000003e-06 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556457+0000", 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92848923299999997 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556535+0000", 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8263000000000006e-05 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52425 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.882541366, 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472399599999999, 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52425 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631916+0000", 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631917+0000", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631922+0000", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4909999999999999e-06 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631935+0000", 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3749e-05 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631943+0000", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6809999999999998e-06 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.631996+0000", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2948999999999997e-05 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632002+0000", 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:43.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556567+0000", 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456459999999996 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556640+0000", 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3183999999999996e-05 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.547 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52429 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.88240021600001, 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92479122199999997, 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52429 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632057+0000", 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632058+0000", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2900000000000002e-07 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632066+0000", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8890000000000002e-06 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632080+0000", 2024-09-18T01:10:43.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3557000000000001e-05 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632087+0000", 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4710000000000001e-06 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632149+0000", 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1830999999999998e-05 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632160+0000", 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556726+0000", 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92456600300000003 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556849+0000", 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122261 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52433 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.88216731200001, 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92472881500000004, 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52433 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632290+0000", 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632291+0000", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8200000000000008e-07 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632296+0000", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632309+0000", 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2849e-05 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632317+0000", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0639999999999994e-06 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632364+0000", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7095000000000002e-05 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.632370+0000", 2024-09-18T01:10:43.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.161e-06 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.556914+0000", 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92454406899999997 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557019+0000", 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104968 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52442 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.87954825, 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92222575699999998, 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52442 2024-09-18T01:10:43.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634909+0000", 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634910+0000", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4199999999999995e-07 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634913+0000", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.275e-06 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634923+0000", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3600000000000002e-06 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634928+0000", 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8799999999999999e-06 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634974+0000", 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6681000000000002e-05 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.634985+0000", 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0407999999999999e-05 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557061+0000", 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92207634599999999 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557135+0000", 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4065000000000003e-05 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:52479 3.10 3:0e3d7777:::rbd_data.117627a3fc4b.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.857209216, 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.90004005099999995, 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 52479 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657248+0000", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657249+0000", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657253+0000", 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4910000000000001e-06 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657261+0000", 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3110000000000006e-06 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657266+0000", 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657325+0000", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9213999999999999e-05 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:29.657337+0000", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1591e-05 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557178+0000", 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89984137799999997 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:30.557288+0000", 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110204 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65883 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 309248~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.906693942, 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035682769000000003, 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.567 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65883 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607764+0000", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8299999999999997e-07 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607766+0000", 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607774+0000", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.109e-06 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607779+0000", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3159999999999996e-06 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643350+0000", 2024-09-18T01:10:43.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035571572000000003 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643392+0000", 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1171999999999998e-05 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643446+0000", 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4806999999999999e-05 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65884 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 331776~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.90667220100001, 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035769902999999999, 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65884 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607785+0000", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607786+0000", 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:10:43.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607788+0000", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607795+0000", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9040000000000003e-06 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607799+0000", 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:43.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643469+0000", 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035669527999999999 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643505+0000", 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6266999999999997e-05 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643555+0000", 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0068000000000002e-05 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65885 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 354304~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.90665179299999, 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036033201000000001, 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65885 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607806+0000", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607808+0000", 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-18T01:10:43.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607814+0000", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607819+0000", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643578+0000", 2024-09-18T01:10:43.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035758692000000002 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643617+0000", 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450999999999999e-05 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643839+0000", 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221506 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65886 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 376832~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.90663213400001, 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036295623999999999, 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65886 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607825+0000", 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607826+0000", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607828+0000", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607840+0000", 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2551999999999999e-05 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607845+0000", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2170000000000002e-06 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643875+0000", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036029765999999998 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.643939+0000", 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3836000000000003e-05 2024-09-18T01:10:43.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644121+0000", 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018196999999999999 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65887 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 399360~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.90659879899999, 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036614652999999997, 2024-09-18T01:10:43.585 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65887 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607859+0000", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607872+0000", 2024-09-18T01:10:43.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2680999999999999e-05 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607880+0000", 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3909999999999998e-06 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607885+0000", 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6820000000000002e-06 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644162+0000", 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036277073 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644260+0000", 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8370999999999997e-05 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644473+0000", 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021305 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65888 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 421888~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.90656426699999, 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036712749000000003, 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65888 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607893+0000", 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607894+0000", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607896+0000", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2369999999999999e-06 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607909+0000", 2024-09-18T01:10:43.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.26e-05 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607924+0000", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.49e-05 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644506+0000", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036582582000000002 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644551+0000", 2024-09-18T01:10:43.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4771000000000002e-05 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644606+0000", 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5194e-05 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65889 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 444416~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.906519377, 2024-09-18T01:10:43.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036807760000000002, 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65889 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607938+0000", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607939+0000", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607941+0000", 2024-09-18T01:10:43.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.362e-06 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607952+0000", 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607957+0000", 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1610000000000002e-06 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644638+0000", 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036681145999999998 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644685+0000", 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7271000000000002e-05 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644746+0000", 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0668000000000003e-05 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65890 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 466944~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.906493002, 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03688781, 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65890 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607965+0000", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607967+0000", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1349999999999999e-06 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607973+0000", 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.071e-06 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607978+0000", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000004e-06 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644760+0000", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036781861999999999 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644799+0000", 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9455000000000001e-05 2024-09-18T01:10:43.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644852+0000", 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3267999999999999e-05 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65891 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 489472~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.906473851, 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037000158999999998, 2024-09-18T01:10:43.603 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65891 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607984+0000", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607986+0000", 2024-09-18T01:10:43.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.917e-06 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607992+0000", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.607997+0000", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5859999999999998e-06 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644883+0000", 2024-09-18T01:10:43.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036886152999999998 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644928+0000", 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5015000000000002e-05 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.644984+0000", 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6066000000000002e-05 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65892 3.a 3:50ff5632:::rbd_data.117658cbc921.0000000000000004:34 [read 512000~2048 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.906455283, 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037110249999999997, 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65892 2024-09-18T01:10:43.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608002+0000", 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:43.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608003+0000", 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608004+0000", 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.75e-06 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608010+0000", 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9050000000000002e-06 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.608015+0000", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645010+0000", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036995706000000003 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645054+0000", 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3538999999999998e-05 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:43.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.645113+0000", 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8578000000000001e-05 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:43.612 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:43.614 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:43.612+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:43.700 INFO:teuthology.orchestra.run.smithi120.stderr:marked in osd.2. 2024-09-18T01:10:43.714 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:43.712+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:43.720 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-18T01:10:43.765 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.765 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:43.765 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:43.765 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.774 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:43.818 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:43.816+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:43.878 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:43.878 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:43.878 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:43.878 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:43.878 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:43.886 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:43.916 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:43.914+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:44.017 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:44.015+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 212.133712686, 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.409784316, 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.40976213499999, 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 74.442257667999996, 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.437285338999999, 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.432392956000001, 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.426978318000003, 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.415404052, 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:44.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.406436358999997, 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:44.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:44.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.406381897000003, 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.081 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:44.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.401110523, 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.087 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:44.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:44.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.401029784000002, 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:44.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.400068785000002, 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.098 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.099 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.400009992999998, 2024-09-18T01:10:44.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:44.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:44.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:44.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:44.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.398846024999997, 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.115 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:44.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:44.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:44.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:44.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:44.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:44.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:44.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.125 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.398481355999998, 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.127 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:44.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.127 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:44.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:44.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:44.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:44.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:44.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.397958819999999, 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:44.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.397561439, 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.144 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:44.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:44.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.395708708999997, 2024-09-18T01:10:44.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:44.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 46.395586518000002, 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:44.154 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:44.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:44.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:44.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:44.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:44.160 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:44.161 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:44.160+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:44.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:44.262 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:44.260+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:44.266 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:44.266 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:44.266 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:44.266 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:44.275 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:44.363 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:44.361+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:44.417 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:44.417 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:44.417 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:44.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:44.467 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:44.465+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:44.564 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:44.562+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 166.846343547, 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.822885456999998, 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809807204999998, 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809786948000003, 2024-09-18T01:10:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809441913000001, 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809134782999998, 2024-09-18T01:10:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809099023999998, 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809060754999997, 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809036282999998, 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.809015621999997, 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808996997999998, 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808977206999998, 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:44.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808958103999998, 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.639 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:44.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:44.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:44.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808940014000001, 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.644 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:44.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:44.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808921914999999, 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.650 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:44.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:44.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:44.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808903586, 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:44.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:44.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.660 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808885220000001, 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:44.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:44.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808866960000003, 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:44.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:44.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:44.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.808848505999997, 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.671 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:44.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.80879882, 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:44.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.682 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:44.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:44.683 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:44.684 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:44.681+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:44.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:10:44.784 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:44.782+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:250301 3.4 3.1196fbc4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e208)", 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:43.854023+0000", 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.97835377300000004, 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.97838175800000005, 2024-09-18T01:10:44.833 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting for peered", 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 250301 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854023+0000", 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854023+0000", 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854025+0000", 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.72e-06 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:44.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854032+0000", 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3660000000000003e-06 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854063+0000", 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0372999999999999e-05 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854086+0000", 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3124000000000001e-05 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854144+0000", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.838e-05 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:43.854153+0000", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:44.573067+0000", 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71891387200000001 2024-09-18T01:10:44.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:44.573080+0000", 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3173000000000001e-05 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-09-18T01:10:44.838 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:44.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:44.856 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:44.856 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:44.874 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:44.874 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:44.885 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:44.884+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:44.891 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:44.891 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:44.907 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:44.907 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:44.924 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:44.924 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:44.941 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:44.941 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:44.958 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:44.958 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:44.960 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:44.960 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:44.969 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:44.974 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:44.975 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:44.986 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:44.984+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:45.077 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.077 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.086 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:45.087 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:45.085+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:45.188 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:45.187+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:45.189 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.189 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:45.290 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:45.288+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:45.339 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.339 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:45.391 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:45.390+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:45.448 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.449 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.457 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:45.501 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:45.499+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:45.559 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.559 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:10:45.592 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:45.591+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:45.599 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-18T01:10:45.693 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:45.691+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:45.708 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.708 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.721 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:45.721 INFO:teuthology.orchestra.run.smithi120.stdout:5717 write 0x365f4ca thru 0x3669bc1 (0xa6f8 bytes) 2024-09-18T01:10:45.736 INFO:teuthology.orchestra.run.smithi120.stdout:5718 punch from 0x1c0ab2c to 0x1c14ab2, (0x9f86 bytes) 2024-09-18T01:10:45.736 INFO:teuthology.orchestra.run.smithi120.stdout:5719 read 0x5f767a9 thru 0x5f792cf (0x2b27 bytes) 2024-09-18T01:10:45.736 INFO:teuthology.orchestra.run.smithi120.stdout:5720 read 0x3184d0f thru 0x318927d (0x456f bytes) 2024-09-18T01:10:45.737 INFO:teuthology.orchestra.run.smithi120.stdout:5721 read 0x5e22652 thru 0x5e2a56b (0x7f1a bytes) 2024-09-18T01:10:45.737 INFO:teuthology.orchestra.run.smithi120.stdout:5722 write 0x3acc7a4 thru 0x3ad01c0 (0x3a1d bytes) 2024-09-18T01:10:45.742 INFO:teuthology.orchestra.run.smithi120.stdout:5723 punch from 0x7ade9a3 to 0x7ae2ac2, (0x411f bytes) 2024-09-18T01:10:45.742 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:45.742 INFO:teuthology.orchestra.run.smithi120.stdout:5726 punch from 0x4f69700 to 0x4f6a6a9, (0xfa9 bytes) 2024-09-18T01:10:45.742 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:45.742 INFO:teuthology.orchestra.run.smithi120.stdout:5728 writesame 0xdc9c3c7 thru 0xdca803d data_size 0x1(0xbc77 bytes) 2024-09-18T01:10:45.772 INFO:teuthology.orchestra.run.smithi120.stdout:5730 trunc from 0xe874328 to 0x806aad8 2024-09-18T01:10:45.793 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:45.791+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:45.818 INFO:teuthology.orchestra.run.smithi120.stdout:5731 writesame 0x4a6b76b thru 0x4a787ff data_size 0x33(0xd095 bytes) 2024-09-18T01:10:45.822 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:45.822 INFO:teuthology.orchestra.run.smithi120.stdout:5733 writesame 0x5e5e362 thru 0x5e65106 data_size 0x1(0x6da5 bytes) 2024-09-18T01:10:45.844 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.844 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.848 INFO:teuthology.orchestra.run.smithi120.stdout:5735 write 0xafdb353 thru 0xafe6e35 (0xbae3 bytes) 2024-09-18T01:10:45.853 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:45.853 INFO:teuthology.orchestra.run.smithi120.stdout:5738 clone 46 order 23 su 4194304 sc 10 2024-09-18T01:10:45.855 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:45.899 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:45.897+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:45.960 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:45.960 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:45.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:45.995 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:45.994+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:46.097 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:46.095+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:46.119 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.119 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:46.212 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:46.210+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:46.277 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.278 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.287 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:46.298 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:46.296+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:46.399 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:46.397+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:46.429 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:46.500 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:46.498+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:46.542 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.542 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:46.635 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:46.634+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:46.695 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.695 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.702 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:46.700+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:46.704 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:46.802 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:46.800+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:46.848 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:46.848 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:46.858 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:46.904 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:46.902+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:46.986 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:47.001 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:47.001 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:47.004 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:47.002+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:47.010 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:47.105 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:47.103+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:47.107 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:47.107 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:47.124 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:47.124 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:47.140 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:47.140 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:47.154 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:47.155 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:47.155 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:47.156 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:47.163 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:47.172 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:47.173 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:47.189 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:47.190 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:47.206 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:47.206 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:47.206 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:47.205+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:47.222 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:47.222 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:47.307 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:47.306+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:47.311 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:47.311 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:47.319 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:47.408 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:47.406+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:47.422 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:47.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:47.422 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:47.423 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:47.423 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:47.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:47.508 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:47.507+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:47.536 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:47.536 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:47.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:47.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:47.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 215.63667658400001, 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:47.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:47.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:47.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.912748214, 2024-09-18T01:10:47.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:47.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:47.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.912726033, 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:47.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.945221566000001, 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:47.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:47.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:47.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.940249237000003, 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:47.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:47.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.935356853999998, 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:47.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:47.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.929942216000001, 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:47.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.918367949999997, 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:47.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.909400257000001, 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:47.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:47.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.909345795, 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:47.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:47.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.904074420999997, 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:47.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:47.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.903993681999999, 2024-09-18T01:10:47.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:47.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:47.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.903032682999999, 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:47.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.902973891000002, 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:47.607 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:47.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.901809923000002, 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:47.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:47.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.901445254000002, 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:47.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:47.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.900922717999997, 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:47.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.900525336999998, 2024-09-18T01:10:47.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:47.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.898672607000002, 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:47.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:47.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.898550415999999, 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:47.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:47.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:47.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:47.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:47.645 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:47.645 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:10:47.647 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:47.647 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:47.644+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:47.747 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:47.745+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:47.759 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:47.759 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:47.759 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:47.759 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:47.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:47.848 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:47.847+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:47.914 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:47.914 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:47.914 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:47.922 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:47.966 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:47.964+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:48.027 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:48.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 170.302741841, 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:10:48.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:10:48.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:48.034 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.279283751000001, 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:48.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.039 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.266205499000002, 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:48.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:10:48.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.266185241999999, 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:10:48.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:10:48.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265840206999997, 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:10:48.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:48.055 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265533077000001, 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:48.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265497318000001, 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:10:48.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:10:48.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:10:48.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265459049, 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265434577000001, 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:48.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265413916, 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:10:48.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:10:48.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:48.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265395292000001, 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:48.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:10:48.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265375501000001, 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:10:48.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:48.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265356398000002, 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.092 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:48.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265338307999997, 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.097 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:10:48.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.102 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265320209000002, 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:10:48.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265301880000003, 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:10:48.108 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:10:48.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:48.113 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265283513999997, 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:48.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:10:48.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265265253999999, 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:10:48.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:10:48.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:10:48.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.2652468, 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:10:48.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.265197114000003, 2024-09-18T01:10:48.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:48.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:10:48.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.134 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.136 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:48.136 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:48.134+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:48.236 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:48.234+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:48.279 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.279 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:48.279 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:48.280 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:48.373 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:48.371+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:48.430 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:48.430 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:48.439 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:48.437+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:48.540 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:48.538+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:48.583 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:48.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.669752719999998, 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.669544688999999, 2024-09-18T01:10:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655508114, 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655484766000001, 2024-09-18T01:10:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655463245, 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655442526000002, 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655421750000002, 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655401605000002, 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655381229, 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655360500999997, 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655336042999998, 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.655315565999999, 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646539369999999, 2024-09-18T01:10:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646514369999998, 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646477214999997, 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646450532000003, 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646430633000001, 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646411426, 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646392413000001, 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.646374139999999, 2024-09-18T01:10:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:48.670 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:48.672 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:48.670+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:48.721 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:48.721 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:48.721 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-18T01:10:48.721 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:10:48.722 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:10:48.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-18T01:10:48.724 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:10:48.772 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:48.771+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:48.833 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.833 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:48.833 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:48.833 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:48.851 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "" 2024-09-18T01:10:48.852 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:48.885 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:48.884+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:48.944 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:48.944 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:48.952 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:48.993 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:48.991+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:49.075 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:49.073+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:49.097 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.435053250999999, 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.098 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:49.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.434999026, 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.103 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:49.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:49.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:49.108 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.434966567999993, 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.434835305000007, 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:49.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:49.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.434365318999994, 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:49.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.433202718999993, 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.123 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:49.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.433095437000006, 2024-09-18T01:10:49.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:49.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.433013778000003, 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.432498390000006, 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.431679152000001, 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.858414065999995, 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.691555174000001, 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.663709785999998, 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.663689486999999, 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.663669224000003, 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.663649371000005, 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.663630213999994, 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 65.279639787999997, 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 65.219680925000006, 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.435177478999996, 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:49.206 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:49.207 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:49.208 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:49.206+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:49.233 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:49.308 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:49.307+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:49.354 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:49.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:49.355 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:49.355 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:49.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:49.370 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:49.370 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:49.389 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:49.389 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:49.405 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:49.405 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:49.409 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:49.408+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:49.422 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:49.439 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:49.439 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:49.456 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:49.456 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:49.469 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:49.469 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:49.473 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:49.473 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:49.479 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:49.490 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:49.510 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:49.509+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:49.597 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:49.597 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:49.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:49.651 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:49.650+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:49.703 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-18T01:10:49.711 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:49.710+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:49.716 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:49.716 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:49.726 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:49.812 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:49.810+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:49.870 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:49.870 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:49.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:49.913 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:49.911+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:49.953 INFO:teuthology.orchestra.run.smithi120.stdout:5739 read 0x746badf thru 0x747ae06 (0xf328 bytes) 2024-09-18T01:10:49.954 INFO:teuthology.orchestra.run.smithi120.stdout:5741 write 0x56ff863 thru 0x5702be2 (0x3380 bytes) 2024-09-18T01:10:49.972 INFO:teuthology.orchestra.run.smithi120.stdout:5742 write 0xeb08a52 thru 0xeb090b0 (0x65f bytes) 2024-09-18T01:10:49.975 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.975 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:5747 write 0xd261d96 thru 0xd2717e6 (0xfa51 bytes) 2024-09-18T01:10:49.979 INFO:teuthology.orchestra.run.smithi120.stdout:5748 punch from 0x54ccafd to 0x54cfaf3, (0x2ff6 bytes) 2024-09-18T01:10:49.979 INFO:teuthology.orchestra.run.smithi120.stdout:5749 punch from 0xd4a2c85 to 0xd4ade4c, (0xb1c7 bytes) 2024-09-18T01:10:49.980 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.980 INFO:teuthology.orchestra.run.smithi120.stdout:5751 read 0xc3affc4 thru 0xc3b77cb (0x7808 bytes) 2024-09-18T01:10:49.980 INFO:teuthology.orchestra.run.smithi120.stdout:5752 read 0x70893a5 thru 0x708c099 (0x2cf5 bytes) 2024-09-18T01:10:49.980 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.980 INFO:teuthology.orchestra.run.smithi120.stdout:5754 writesame 0x847fd68 thru 0x848a567 data_size 0xa8(0xa800 bytes) 2024-09-18T01:10:49.982 INFO:teuthology.orchestra.run.smithi120.stdout:5757 read 0xb518c17 thru 0xb51c55f (0x3949 bytes) 2024-09-18T01:10:49.982 INFO:teuthology.orchestra.run.smithi120.stdout:5759 write 0xe09fa22 thru 0xe0a287b (0x2e5a bytes) 2024-09-18T01:10:49.984 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:49.984 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:49.986 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.986 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:49.986 INFO:teuthology.orchestra.run.smithi120.stdout:5764 write 0x91fade6 thru 0x9201484 (0x669f bytes) 2024-09-18T01:10:49.990 INFO:teuthology.orchestra.run.smithi120.stdout:5765 read 0x3950bff thru 0x3953905 (0x2d07 bytes) 2024-09-18T01:10:49.991 INFO:teuthology.orchestra.run.smithi120.stdout:5767 trunc from 0xeb090b1 to 0xda22e5e 2024-09-18T01:10:49.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:50.002 INFO:teuthology.orchestra.run.smithi120.stdout:5768 trunc from 0xda22e5e to 0xb5a2b42 2024-09-18T01:10:50.015 INFO:teuthology.orchestra.run.smithi120.stdout:5769 write 0x4433e2f thru 0x44362b4 (0x2486 bytes) 2024-09-18T01:10:50.016 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:50.015+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:50.020 INFO:teuthology.orchestra.run.smithi120.stdout:5772 trunc from 0xb5a2b42 to 0x9dbc474 2024-09-18T01:10:50.034 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.035 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.037 INFO:teuthology.orchestra.run.smithi120.stdout:5775 trunc from 0x9dbc474 to 0xbeab0e4 2024-09-18T01:10:50.055 INFO:teuthology.orchestra.run.smithi120.stdout:5776 trunc from 0xbeab0e4 to 0x11eb380 2024-09-18T01:10:50.083 INFO:teuthology.orchestra.run.smithi120.stdout:5777 read 0xaf4e49 thru 0xafc248 (0x7400 bytes) 2024-09-18T01:10:50.097 INFO:teuthology.orchestra.run.smithi120.stdout:5778 trunc from 0x11eb380 to 0xcf7592a 2024-09-18T01:10:50.100 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.100 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.105 INFO:teuthology.orchestra.run.smithi120.stdout:5779 read 0xbb453d7 thru 0xbb4bc44 (0x686e bytes) 2024-09-18T01:10:50.105 INFO:teuthology.orchestra.run.smithi120.stdout:5783 write 0x1f0b62e thru 0x1f0b8f7 (0x2ca bytes) 2024-09-18T01:10:50.109 INFO:teuthology.orchestra.run.smithi120.stdout:5784 read 0xabf77c thru 0xac78ff (0x8184 bytes) 2024-09-18T01:10:50.109 INFO:teuthology.orchestra.run.smithi120.stdout:5785 writesame 0x56325fa thru 0x56387a7 data_size 0x1(0x61ae bytes) 2024-09-18T01:10:50.110 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:50.128 INFO:teuthology.orchestra.run.smithi120.stdout:5786 write 0x6c0c1e4 thru 0x6c12a60 (0x687d bytes) 2024-09-18T01:10:50.131 INFO:teuthology.orchestra.run.smithi120.stdout:5787 read 0xa72fdad thru 0xa73cc56 (0xceaa bytes) 2024-09-18T01:10:50.131 INFO:teuthology.orchestra.run.smithi120.stdout:5788 writesame 0x9bdfd7b thru 0x9be0904 data_size 0x7(0xb8a bytes) 2024-09-18T01:10:50.146 INFO:teuthology.orchestra.run.smithi120.stdout:5789 trunc from 0xcf7592a to 0x427abba 2024-09-18T01:10:50.154 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:50.153+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:50.180 INFO:teuthology.orchestra.run.smithi120.stdout:5791 writesame 0xc13e3f0 thru 0xc14365c data_size 0x1(0x526d bytes) 2024-09-18T01:10:50.180 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.180 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.180 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.181 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.181 INFO:teuthology.orchestra.run.smithi120.stdout:5800 read 0xab25d98 thru 0xab35add (0xfd46 bytes) 2024-09-18T01:10:50.181 INFO:teuthology.orchestra.run.smithi120.stdout:5801 punch from 0x2374c50 to 0x237bd04, (0x70b4 bytes) 2024-09-18T01:10:50.181 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.181 INFO:teuthology.orchestra.run.smithi120.stdout:5805 write 0xbe72886 thru 0xbe76478 (0x3bf3 bytes) 2024-09-18T01:10:50.195 INFO:teuthology.orchestra.run.smithi120.stdout:5807 trunc from 0xc14365d to 0x4819f60 2024-09-18T01:10:50.208 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.208 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.214 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.214 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.218 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:50.217+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:50.224 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:50.224 INFO:teuthology.orchestra.run.smithi120.stdout:5810 writesame 0xba7aa9e thru 0xba8064c data_size 0x7(0x5baf bytes) 2024-09-18T01:10:50.234 INFO:teuthology.orchestra.run.smithi120.stdout:5811 write 0xe2927ec thru 0xe29ab20 (0x8335 bytes) 2024-09-18T01:10:50.237 INFO:teuthology.orchestra.run.smithi120.stdout:5812 punch from 0x4958f97 to 0x4968249, (0xf2b2 bytes) 2024-09-18T01:10:50.237 INFO:teuthology.orchestra.run.smithi120.stdout:5814 punch from 0x7e2f32f to 0x7e367fd, (0x74ce bytes) 2024-09-18T01:10:50.238 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:50.239 INFO:teuthology.orchestra.run.smithi120.stdout:5817 trunc from 0xe29ab21 to 0xce63355 2024-09-18T01:10:50.248 INFO:teuthology.orchestra.run.smithi120.stdout:5818 writesame 0x5e45a90 thru 0x5e55520 data_size 0x1(0xfa91 bytes) 2024-09-18T01:10:50.290 INFO:teuthology.orchestra.run.smithi120.stdout:5819 write 0xe28e36d thru 0xe299fb6 (0xbc4a bytes) 2024-09-18T01:10:50.291 INFO:teuthology.orchestra.run.smithi120.stdout:5821 read 0x2fa67ab thru 0x2fa7918 (0x116e bytes) 2024-09-18T01:10:50.298 INFO:teuthology.orchestra.run.smithi120.stdout:5822 trunc from 0xe299fb7 to 0x839ddb0 2024-09-18T01:10:50.314 INFO:teuthology.orchestra.run.smithi120.stdout:5823 punch from 0x5cf37ac to 0x5cfcb13, (0x9367 bytes) 2024-09-18T01:10:50.314 INFO:teuthology.orchestra.run.smithi120.stdout:5825 read 0x690ca53 thru 0x691692a (0x9ed8 bytes) 2024-09-18T01:10:50.318 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:50.317+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:50.322 INFO:teuthology.orchestra.run.smithi120.stdout:5826 writesame 0x9e434ef thru 0x9e4c825 data_size 0xd(0x9337 bytes) 2024-09-18T01:10:50.322 INFO:teuthology.orchestra.run.smithi120.stdout:5827 writesame 0x8d29ad5 thru 0x8d302f6 data_size 0x9(0x6822 bytes) 2024-09-18T01:10:50.326 INFO:teuthology.orchestra.run.smithi120.stdout:5828 punch from 0x676a005 to 0x6773d1b, (0x9d16 bytes) 2024-09-18T01:10:50.326 INFO:teuthology.orchestra.run.smithi120.stdout:5829 read 0x3ad0f94 thru 0x3ad96df (0x874c bytes) 2024-09-18T01:10:50.326 INFO:teuthology.orchestra.run.smithi120.stdout:5830 read 0x7dff2b1 thru 0x7e09e38 (0xab88 bytes) 2024-09-18T01:10:50.327 INFO:teuthology.orchestra.run.smithi120.stdout:5831 read 0x1da0ccb thru 0x1da41ab (0x34e1 bytes) 2024-09-18T01:10:50.338 INFO:teuthology.orchestra.run.smithi120.stdout:5832 writesame 0xdebb7af thru 0xdebc377 data_size 0x1(0xbc9 bytes) 2024-09-18T01:10:50.338 INFO:teuthology.orchestra.run.smithi120.stdout:5836 write 0xbc3330b thru 0xbc3a7b7 (0x74ad bytes) 2024-09-18T01:10:50.341 INFO:teuthology.orchestra.run.smithi120.stdout:5837 clone 47 order 24 su 524288 sc 9 2024-09-18T01:10:50.369 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.370 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:10:50.419 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:50.418+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:50.520 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:50.519+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:50.522 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.522 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:10:50.621 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:50.620+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:50.634 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.634 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:10:50.721 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:50.720+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:50.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.752 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.763 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:50.823 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:50.822+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:50.907 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:50.907 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:50.916 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:50.960 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:50.959+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:51.019 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:51.019 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:51.024 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:51.023+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:51.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:51.124 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:51.123+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:51.173 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:51.173 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:51.183 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:51.227 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:51.226+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:51.286 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:51.286 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:51.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:51.335 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:51.334+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:51.427 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:51.426+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:51.436 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:51.436 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:51.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:51.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:10:51.546 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:51.545+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:51.549 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:51.549 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:51.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:51.624 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:51.624 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:51.628 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:51.628+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:51.641 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:51.641 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:51.658 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:51.658 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:51.674 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:51.674 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:51.692 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:51.692 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:51.703 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:51.703 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:51.703 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:51.703 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:51.707 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:51.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:51.725 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-18T01:10:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:10:51.733 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:10:51.733 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:51.732+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:51.754 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:51.830 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:51.829+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:51.845 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:51.846 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:51.846 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:51.854 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:51.931 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:51.930+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.045163608999999, 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.963 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.044955578, 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030919003000001, 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.968 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030895655000002, 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.972 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:10:51.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030874134000001, 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:10:51.976 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:10:51.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030853415000003, 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:10:51.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030832639000003, 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.986 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:10:51.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030812494000003, 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.994 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030792118000001, 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:51.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:51.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:10:51.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030771389999998, 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:10:52.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:10:52.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030746932, 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:10:52.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:10:52.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.030726455, 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:10:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021950259, 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:10:52.012 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:10:52.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021925259, 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:10:52.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:10:52.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021888103999999, 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:10:52.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021861420999997, 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021841522000003, 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021822315000001, 2024-09-18T01:10:52.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:10:52.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:10:52.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021803302000002, 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:10:52.036 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.021785029, 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:52.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.044 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.045 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:52.046 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:52.044+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:52.146 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:52.145+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:52.189 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:52.190 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:52.247 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:52.246+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:52.301 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:52.302 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:52.302 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.310 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:52.348 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:52.347+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:52.448 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:52.447+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:52.460 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:52.460 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:52.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.797827884, 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:10:52.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:10:52.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.797773659000001, 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:10:52.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.797741200999994, 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:10:52.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:10:52.475 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.797609937999994, 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.476 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:10:52.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:10:52.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.480 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.797139951999995, 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:10:52.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.795977351999994, 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:10:52.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.795870070000007, 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:10:52.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.795788411000004, 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:10:52.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:10:52.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.795273022999993, 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:10:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.794453785000002, 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:10:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:10:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:10:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.221188698999995, 2024-09-18T01:10:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:10:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:10:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:10:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.054329807000002, 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:10:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.026484418999999, 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:10:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:10:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:10:52.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.02646412, 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:10:52.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.026443857000004, 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:10:52.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:10:52.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:10:52.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.026424004000006, 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:10:52.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.026404846999995, 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:10:52.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:10:52.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.557 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.642414420999998, 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:10:52.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:10:52.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.582455558000007, 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:52.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:10:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:10:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.797952111999997, 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:10:52.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:52.575 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:52.576 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:52.578 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:52.576+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:52.677 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:52.677+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:52.721 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:52.721 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:52.721 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:10:52.721 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.730 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:52.778 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:52.777+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:52.879 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:10:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:10:52.879 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:10:52.879 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:52.880 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:52.878+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:52.889 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-18T01:10:52.980 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:10:52.979+0000 7f7bad0b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33240) UID: 0 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379337096, 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.042 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:10:53.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.046 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379317899999997, 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:10:53.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:10:53.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.051 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379299539000002, 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:10:53.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:10:53.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.056 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379281435000003, 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:10:53.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:53.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379262838000002, 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:10:53.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:10:53.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:10:53.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:10:53.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379239837999997, 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:10:53.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379221166000001, 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:10:53.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:10:53.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379201162999998, 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379182923000002, 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:53.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:10:53.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:10:53.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379164508999999, 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.089 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:10:53.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.095 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379142481999999, 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.096 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.100 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374890702999998, 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:10:53.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374880251, 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374868560000003, 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:10:53.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:10:53.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:10:53.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374855263000001, 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:10:53.114 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:10:53.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.119 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374839192000003, 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:10:53.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:10:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374822534000003, 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.126 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:10:53.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.130 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374803239000002, 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:10:53.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374775008999997, 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:53.139 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.374753884999997, 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:10:53.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:10:53.144 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:10:53.144 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:10:53.145 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:53.146 INFO:teuthology.orchestra.run.smithi120.stdout:5838 writesame 0xbb2fe7 thru 0xbb75d2 data_size 0x64(0x45ec bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5839 write 0x6469b77 thru 0x646b4de (0x1968 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5841 read 0xb480678 thru 0xb487df2 (0x777b bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5842 write 0xaa34f00 thru 0xaa41d15 (0xce16 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5843 read 0xbf45bfb thru 0xbf47115 (0x151b bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5844 writesame 0x4e100b2 thru 0x4e13a63 data_size 0x23(0x39b2 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5847 write 0x81c4ff4 thru 0x81d4f59 (0xff66 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5850 read 0x31d9b2e thru 0x31dfdb1 (0x6284 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5851 punch from 0x8ff0f2b to 0x8ff51af, (0x4284 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5853 write 0x4c8b3c1 thru 0x4c92a29 (0x7669 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5855 read 0xd5172cc thru 0xd52198c (0xa6c1 bytes) 2024-09-18T01:10:53.147 INFO:teuthology.orchestra.run.smithi120.stdout:5856 clone 48 order 22 su 2097152 sc 3 2024-09-18T01:10:53.148 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:53.144+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:53.247 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:53.245+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:53.258 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:53.258 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:53.258 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:53.267 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:53.347 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:53.346+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:53.370 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:53.370 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:53.370 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:53.370 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:53.379 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:53.448 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:53.447+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:53.523 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:53.523 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 175.79852344400001, 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:53.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:10:53.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:10:53.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767496921999999, 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:10:53.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767487344000003, 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:10:53.541 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767473318, 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767444545000004, 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:10:53.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767416849, 2024-09-18T01:10:53.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:53.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:10:53.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:10:53.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767382400999999, 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:10:53.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:53.564 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767352555000002, 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:10:53.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767330364000003, 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:10:53.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767293457999997, 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:10:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767272343000002, 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.767241818000002, 2024-09-18T01:10:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:10:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761761563999997, 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:10:53.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:10:53.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761707704999999, 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:10:53.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761630283000002, 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:10:53.601 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:10:53.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.606 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761600168999998, 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:10:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:10:53.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761550251000003, 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:10:53.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:10:53.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:10:53.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761516663999998, 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:10:53.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:10:53.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761492130000001, 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.622 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:10:53.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:53.627 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761306847999997, 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:10:53.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:10:53.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:10:53.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:53.632 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:53.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:53.634 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:53.633+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:53.735 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:53.734+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:53.740 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:53.749 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:53.754 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:10:53.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:10:53.835 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:53.834+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:53.861 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:53.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:10:53.861 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:10:53.861 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T01:10:53.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:10:53.890 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T01:10:53.890 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-18T01:10:53.891 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:10:53.891 DEBUG:teuthology.orchestra.run.smithi162:> type ceph-objectstore-tool 2024-09-18T01:10:53.894 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:53.894 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:53.895 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:53.895 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:53.904 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:53.909 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T01:10:53.909 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-18T01:10:53.921 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:10:53.921 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:10:53.936 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:53.935+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:53.939 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:10:53.939 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:10:53.955 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:10:53.955 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:10:53.971 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:10:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:10:53.988 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:10:53.988 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:10:54.005 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:10:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:10:54.037 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:54.036+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:54.055 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:54.055 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:54.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:54.139 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:54.138+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:54.167 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:54.167 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:54.167 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:54.167 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:54.176 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:54.240 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:54.239+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:54.280 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:54.280 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:54.280 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:54.280 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:54.280 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:54.289 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:54.340 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:54.339+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 222.496242795, 2024-09-18T01:10:54.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.772314425, 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:54.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.772292244, 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:54.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.804787777000001, 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:54.412 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:54.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:54.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:54.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.799815447999997, 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.419 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.794923064999999, 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.424 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:54.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.789508427000001, 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:54.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.777934160999997, 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.438 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.768966468000002, 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:54.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.768912006000001, 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:54.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.763640631999998, 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:54.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:54.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:54.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.763559893, 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:54.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.762598894, 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:54.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.462 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.762540102000003, 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:54.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:54.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761376134000002, 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.468 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:54.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:54.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.761011465000003, 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:54.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:54.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.760488928999997, 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:54.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:54.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:54.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.760091547999998, 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.758238818000002, 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:54.488 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.758116627, 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:54.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:54.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:54.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:54.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:54.498 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:54.499 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:54.500 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:54.498+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:54.572 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:10:54.600 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:54.599+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:54.617 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:54.617 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:54.628 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:54.712 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:54.711+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:54.772 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:54.772 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:54.781 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:54.802 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:54.801+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:54.902 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:54.901+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:54.923 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:54.923 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:54.932 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:55.004 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:55.003+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:55.075 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.075 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:55.128 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:55.127+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:55.187 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.187 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.196 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:55.205 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:55.204+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:55.306 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:55.305+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:55.339 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.339 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.347 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:55.407 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:55.406+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:55.450 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.450 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:55.508 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:55.507+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:55.608 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.608 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.609 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:55.607+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:55.618 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:55.709 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:55.708+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:55.721 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.721 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.730 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:55.810 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:55.809+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:55.834 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.835 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.843 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:55.928 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:55.927+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:55.987 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:55.987 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:55.996 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:56.013 INFO:teuthology.orchestra.run.smithi162.stdout:3.a 2024-09-18T01:10:56.013 INFO:teuthology.orchestra.run.smithi162.stdout:3.1e 2024-09-18T01:10:56.014 INFO:teuthology.orchestra.run.smithi162.stdout:3.10 2024-09-18T01:10:56.014 INFO:teuthology.orchestra.run.smithi162.stdout:2.7 2024-09-18T01:10:56.014 INFO:teuthology.orchestra.run.smithi162.stdout:3.3 2024-09-18T01:10:56.014 INFO:teuthology.orchestra.run.smithi162.stdout:2.6 2024-09-18T01:10:56.037 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:56.036+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:56.113 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:56.112+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:56.140 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.140 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:56.214 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:56.213+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:56.253 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.253 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.263 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:56.314 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:56.313+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:56.366 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.367 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.375 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:56.415 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:56.415+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:56.517 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:56.516+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:56.519 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.519 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.528 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:56.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 3.10 --file /var/log/ceph/exp.3.10.6 2024-09-18T01:10:56.589 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:10:56.633 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:56.632+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:56.637 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.637 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.646 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:56.719 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:56.718+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:56.741 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-18T01:10:56.753 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.753 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.762 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:56.820 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:56.819+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:56.868 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:56.868 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:56.876 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:56.920 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:10:56.919+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:10:56.987 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:56.988 INFO:teuthology.orchestra.run.smithi120.stdout:5860 writesame 0xd3ad30f thru 0xd3b4954 data_size 0x1(0x7646 bytes) 2024-09-18T01:10:57.011 INFO:teuthology.orchestra.run.smithi120.stdout:5861 writesame 0xe767d25 thru 0xe76eee3 data_size 0x1(0x71bf bytes) 2024-09-18T01:10:57.011 INFO:teuthology.orchestra.run.smithi120.stdout:5862 read 0x3cbe4d8 thru 0x3cc22a7 (0x3dd0 bytes) 2024-09-18T01:10:57.011 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.011 INFO:teuthology.orchestra.run.smithi120.stdout:5865 punch from 0xcfd5e24 to 0xcfe5e09, (0xffe5 bytes) 2024-09-18T01:10:57.012 INFO:teuthology.orchestra.run.smithi120.stdout:5866 write 0x7364cc thru 0x73aa66 (0x459b bytes) 2024-09-18T01:10:57.013 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.013 INFO:teuthology.orchestra.run.smithi120.stdout:5868 read 0x5502399 thru 0x550a8e9 (0x8551 bytes) 2024-09-18T01:10:57.013 INFO:teuthology.orchestra.run.smithi120.stdout:5869 writesame 0x1ad15ba thru 0x1add877 data_size 0x15(0xc2be bytes) 2024-09-18T01:10:57.017 INFO:teuthology.orchestra.run.smithi120.stdout:5870 read 0x5321f0b thru 0x532da20 (0xbb16 bytes) 2024-09-18T01:10:57.020 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:57.019+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:57.024 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:57.024 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:10:57.024 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:57.032 INFO:teuthology.orchestra.run.smithi120.stdout:5871 trunc from 0xe76eee4 to 0x2e0f688 2024-09-18T01:10:57.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:57.078 INFO:teuthology.orchestra.run.smithi120.stdout:5872 write 0xb1b2c64 thru 0xb1c0ad5 (0xde72 bytes) 2024-09-18T01:10:57.082 INFO:teuthology.orchestra.run.smithi120.stdout:5873 trunc from 0xb1c0ad6 to 0xbac3423 2024-09-18T01:10:57.087 INFO:teuthology.orchestra.run.smithi120.stdout:5876 punch from 0x971d141 to 0x9720b4b, (0x3a0a bytes) 2024-09-18T01:10:57.087 INFO:teuthology.orchestra.run.smithi120.stdout:5877 read 0x894726b thru 0x8955472 (0xe208 bytes) 2024-09-18T01:10:57.087 INFO:teuthology.orchestra.run.smithi120.stdout:5880 writesame 0x7c8b546 thru 0x7c936bb data_size 0x2(0x8176 bytes) 2024-09-18T01:10:57.096 INFO:teuthology.orchestra.run.smithi120.stdout:5881 read 0xad3ddd thru 0xae29f9 (0xec1d bytes) 2024-09-18T01:10:57.097 INFO:teuthology.orchestra.run.smithi120.stdout:5882 read 0x429d164 thru 0x42a3119 (0x5fb6 bytes) 2024-09-18T01:10:57.098 INFO:teuthology.orchestra.run.smithi120.stdout:5883 trunc from 0xbac3423 to 0xcc8b25c 2024-09-18T01:10:57.104 INFO:teuthology.orchestra.run.smithi120.stdout:5886 write 0x42aa9e2 thru 0x42aac46 (0x265 bytes) 2024-09-18T01:10:57.109 INFO:teuthology.orchestra.run.smithi120.stdout:5887 trunc from 0xcc8b25c to 0xaf933c6 2024-09-18T01:10:57.122 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:57.121+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:57.128 INFO:teuthology.orchestra.run.smithi120.stdout:5888 trunc from 0xaf933c6 to 0xa9e702 2024-09-18T01:10:57.145 INFO:teuthology.orchestra.run.smithi120.stdout:5890 trunc from 0xa9e702 to 0x1796d27 2024-09-18T01:10:57.163 INFO:teuthology.orchestra.run.smithi120.stdout:5892 write 0x6bcf45d thru 0x6bd3d57 (0x48fb bytes) 2024-09-18T01:10:57.167 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.167 INFO:teuthology.orchestra.run.smithi120.stdout:5894 read 0x2e62f36 thru 0x2e6f43b (0xc506 bytes) 2024-09-18T01:10:57.179 INFO:teuthology.orchestra.run.smithi120.stdout:5895 writesame 0xcaa70dd thru 0xcab1a36 data_size 0x35(0xa95a bytes) 2024-09-18T01:10:57.180 INFO:teuthology.orchestra.run.smithi120.stdout:5896 punch from 0x84a17c3 to 0x84b1597, (0xfdd4 bytes) 2024-09-18T01:10:57.181 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:10:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:10:57.181 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:10:57.181 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:57.186 INFO:teuthology.orchestra.run.smithi120.stdout:5897 write 0xdd97648 thru 0xdd9f568 (0x7f21 bytes) 2024-09-18T01:10:57.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:10:57.191 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.191 INFO:teuthology.orchestra.run.smithi120.stdout:5899 read 0x1e922ff thru 0x1ea1208 (0xef0a bytes) 2024-09-18T01:10:57.191 INFO:teuthology.orchestra.run.smithi120.stdout:5900 write 0x6544585 thru 0x654d002 (0x8a7e bytes) 2024-09-18T01:10:57.192 INFO:teuthology.orchestra.run.smithi120.stdout:5901 read 0x4b85360 thru 0x4b94845 (0xf4e6 bytes) 2024-09-18T01:10:57.193 INFO:teuthology.orchestra.run.smithi120.stdout:5902 trunc from 0xdd9f569 to 0xd2ea6f8 2024-09-18T01:10:57.212 INFO:teuthology.orchestra.run.smithi120.stdout:5903 trunc from 0xd2ea6f8 to 0x5a8acc4 2024-09-18T01:10:57.224 INFO:teuthology.orchestra.run.smithi120.stdout:5905 write 0x36a0965 thru 0x36ae0ac (0xd748 bytes) 2024-09-18T01:10:57.234 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:57.233+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:57.242 INFO:teuthology.orchestra.run.smithi120.stdout:5906 write 0xd604e32 thru 0xd60ee89 (0xa058 bytes) 2024-09-18T01:10:57.246 INFO:teuthology.orchestra.run.smithi120.stdout:5907 read 0x6ce3af2 thru 0x6cece26 (0x9335 bytes) 2024-09-18T01:10:57.246 INFO:teuthology.orchestra.run.smithi120.stdout:5908 punch from 0x6f681ba to 0x6f7540b, (0xd251 bytes) 2024-09-18T01:10:57.246 INFO:teuthology.orchestra.run.smithi120.stdout:5909 write 0x83bdc60 thru 0x83c4084 (0x6425 bytes) 2024-09-18T01:10:57.249 INFO:teuthology.orchestra.run.smithi120.stdout:5914 punch from 0x207d38 to 0x20d1a9, (0x5471 bytes) 2024-09-18T01:10:57.249 INFO:teuthology.orchestra.run.smithi120.stdout:5915 write 0xb55bcb thru 0xb56e3e (0x1274 bytes) 2024-09-18T01:10:57.250 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.251 INFO:teuthology.orchestra.run.smithi120.stdout:5917 punch from 0xb95d335 to 0xb95eac8, (0x1793 bytes) 2024-09-18T01:10:57.256 INFO:teuthology.orchestra.run.smithi120.stdout:5920 trunc from 0xd60ee8a to 0x915bba7 2024-09-18T01:10:57.267 INFO:teuthology.orchestra.run.smithi120.stdout:5922 punch from 0x616b5d2 to 0x616d5c0, (0x1fee bytes) 2024-09-18T01:10:57.267 INFO:teuthology.orchestra.run.smithi120.stdout:5923 punch from 0x7e077f2 to 0x7e140a2, (0xc8b0 bytes) 2024-09-18T01:10:57.267 INFO:teuthology.orchestra.run.smithi120.stdout:5924 trunc from 0x915bba7 to 0x8d43e05 2024-09-18T01:10:57.282 INFO:teuthology.orchestra.run.smithi120.stdout:5926 trunc from 0x8d43e05 to 0x1e8766 2024-09-18T01:10:57.298 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:10:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:10:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:10:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 225.39872157400001, 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:10:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:10:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:10:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.674793204, 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:10:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:10:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.67477102300001, 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:10:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:10:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 87.707266555999993, 2024-09-18T01:10:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:10:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:10:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.702294227000003, 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:10:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.697401843999998, 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:10:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.691987206, 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:10:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.680412939999997, 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:10:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.671445247000001, 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:10:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:10:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.671390785, 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:10:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:10:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:57.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.666119410999997, 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:57.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.666038671999999, 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:10:57.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:10:57.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.665077672999999, 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:10:57.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.665018881000002, 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.370 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:10:57.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:10:57.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.663854913000002, 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:10:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.663490244000002, 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:10:57.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.662967707999996, 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.387 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:10:57.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.662570326999997, 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:10:57.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.660717597000001, 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:10:57.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 59.660595405999999, 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:10:57.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:10:57.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:10:57.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:10:57.408 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:10:57.410 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:57.408+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:57.411 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:5928 read 0xbbd90 thru 0xc1197 (0x5408 bytes) 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:5929 writesame 0x79b80f3 thru 0x79bb8fd data_size 0x1(0x380b bytes) 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:5931 read 0x4b59f5f thru 0x4b5aff6 (0x1098 bytes) 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:5932 write 0x6659c2b thru 0x6664f8e (0xb364 bytes) 2024-09-18T01:10:57.412 INFO:teuthology.orchestra.run.smithi120.stdout:5933 read 0x6c325c9 thru 0x6c32c3a (0x672 bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5935 writesame 0xcbaa409 thru 0xcbb420e data_size 0x2(0x9e06 bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5936 write 0x3e93a8 thru 0x3ec146 (0x2d9f bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5937 punch from 0xa3e5be6 to 0xa3ef67f, (0x9a99 bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5939 write 0xebeb2d0 thru 0xebf0b5c (0x588d bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5940 trunc from 0xebf0b5d to 0x7d6cc4b 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5941 trunc from 0x7d6cc4b to 0x51ffdd2 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5942 punch from 0x491e830 to 0x492d98e, (0xf15e bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5946 write 0x5e22a24 thru 0x5e2cf1b (0xa4f8 bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5947 writesame 0xd995fc4 thru 0xd996b83 data_size 0x8(0xbc0 bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5948 read 0xc1bd436 thru 0xc1c0c24 (0x37ef bytes) 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.413 INFO:teuthology.orchestra.run.smithi120.stdout:5950 write 0x6f709be thru 0x6f73b42 (0x3185 bytes) 2024-09-18T01:10:57.414 INFO:teuthology.orchestra.run.smithi120.stdout:5952 read 0x6354aa6 thru 0x6361f1d (0xd478 bytes) 2024-09-18T01:10:57.414 INFO:teuthology.orchestra.run.smithi120.stdout:5953 writesame 0xd5bdcce thru 0xd5c32c6 data_size 0x1(0x55f9 bytes) 2024-09-18T01:10:57.424 INFO:teuthology.orchestra.run.smithi120.stdout:5955 write 0x3b8dbe1 thru 0x3b9c486 (0xe8a6 bytes) 2024-09-18T01:10:57.428 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.428 INFO:teuthology.orchestra.run.smithi120.stdout:5958 read 0x4649a69 thru 0x4655ccf (0xc267 bytes) 2024-09-18T01:10:57.428 INFO:teuthology.orchestra.run.smithi120.stdout:5959 read 0x18b9766 thru 0x18c39e8 (0xa283 bytes) 2024-09-18T01:10:57.428 INFO:teuthology.orchestra.run.smithi120.stdout:5960 writesame 0xcad2191 thru 0xcad52e7 data_size 0x1(0x3157 bytes) 2024-09-18T01:10:57.437 INFO:teuthology.orchestra.run.smithi120.stdout:5962 read 0x401702b thru 0x401a4d1 (0x34a7 bytes) 2024-09-18T01:10:57.448 INFO:teuthology.orchestra.run.smithi120.stdout:5964 trunc from 0xd996b84 to 0x408a02d 2024-09-18T01:10:57.460 INFO:teuthology.orchestra.run.smithi120.stdout:5966 trunc from 0x408a02d to 0x4d75fa6 2024-09-18T01:10:57.479 INFO:teuthology.orchestra.run.smithi120.stdout:5967 writesame 0xae64d8a thru 0xae678fc data_size 0x7(0x2b73 bytes) 2024-09-18T01:10:57.480 INFO:teuthology.orchestra.run.smithi120.stdout:5970 trunc from 0xae678fd to 0xa1bc12f 2024-09-18T01:10:57.486 INFO:teuthology.orchestra.run.smithi120.stdout:5971 read 0x5a79a81 thru 0x5a7fce1 (0x6261 bytes) 2024-09-18T01:10:57.488 INFO:teuthology.orchestra.run.smithi120.stdout:5972 trunc from 0xa1bc12f to 0x839e5b6 2024-09-18T01:10:57.494 INFO:teuthology.orchestra.run.smithi120.stdout:5974 read 0xec2d8f thru 0xec7cf5 (0x4f67 bytes) 2024-09-18T01:10:57.494 INFO:teuthology.orchestra.run.smithi120.stdout:5977 read 0x4d2a59b thru 0x4d309eb (0x6451 bytes) 2024-09-18T01:10:57.496 INFO:teuthology.orchestra.run.smithi120.stdout:5978 trunc from 0x839e5b6 to 0x6afe501 2024-09-18T01:10:57.502 INFO:teuthology.orchestra.run.smithi120.stdout:5981 read 0x400f7ef thru 0x4018299 (0x8aab bytes) 2024-09-18T01:10:57.502 INFO:teuthology.orchestra.run.smithi120.stdout:5982 writesame 0x6464ce2 thru 0x64679a3 data_size 0x11(0x2cc2 bytes) 2024-09-18T01:10:57.505 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.510 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:57.509+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:57.514 INFO:teuthology.orchestra.run.smithi120.stdout:5985 write 0x9a2b877 thru 0x9a2d3b5 (0x1b3f bytes) 2024-09-18T01:10:57.525 INFO:teuthology.orchestra.run.smithi120.stdout:5989 write 0xab088a6 thru 0xab10a6b (0x81c6 bytes) 2024-09-18T01:10:57.530 INFO:teuthology.orchestra.run.smithi120.stdout:5990 trunc from 0xab10a6c to 0xd61f35d 2024-09-18T01:10:57.532 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:57.532 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:57.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:10:57.545 INFO:teuthology.orchestra.run.smithi120.stdout:5991 writesame 0xdbcd985 thru 0xdbd77c7 data_size 0x6f(0x9e43 bytes) 2024-09-18T01:10:57.550 INFO:teuthology.orchestra.run.smithi120.stdout:5993 trunc from 0xdbd77c8 to 0x9104c0f 2024-09-18T01:10:57.569 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.569 INFO:teuthology.orchestra.run.smithi120.stdout:compare and write disabled 2024-09-18T01:10:57.569 INFO:teuthology.orchestra.run.smithi120.stdout:5996 read 0x776097 thru 0x77f545 (0x94af bytes) 2024-09-18T01:10:57.580 INFO:teuthology.orchestra.run.smithi120.stdout:5997 write 0xdaab272 thru 0xdaaea0a (0x3799 bytes) 2024-09-18T01:10:57.585 INFO:teuthology.orchestra.run.smithi120.stdout:5998 write 0x1dcd720 thru 0x1dd4ac6 (0x73a7 bytes) 2024-09-18T01:10:57.587 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:10:57.594 INFO:teuthology.orchestra.run.smithi120.stdout:5999 punch from 0x37dba7c to 0x37dc94f, (0xed3 bytes) 2024-09-18T01:10:57.594 INFO:teuthology.orchestra.run.smithi120.stdout:6000 write 0x4d1ce8a thru 0x4d21d8e (0x4f05 bytes) 2024-09-18T01:10:57.611 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:57.610+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:57.618 INFO:teuthology.orchestra.run.smithi120.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-18T01:10:57.667 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:57.667 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:57.679 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:10:57.720 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:57.719+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:57.814 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:57.813+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:57.835 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:10:57.919 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:57.918+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:57.987 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:57.987 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:57.996 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:10:58.038 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:58.037+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:58.117 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:58.116+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:58.140 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.140 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:10:58.234 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:58.233+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:58.292 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.292 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.301 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:10:58.342 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:10:58.341+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:10:58.420 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:58.419+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:58.444 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.444 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.453 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:10:58.476 INFO:teuthology.orchestra.run.smithi162.stderr:Exporting 3.10 info 3.10( v 214'485 (0'0,214'485] local-lis/les=82/83 n=50 ec=23/23 lis/c=82/82 les/c/f=83/83/0 sis=82) 2024-09-18T01:10:58.487 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head# 2024-09-18T01:10:58.490 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:086f8398:::rbd_data.1176cffa61f8.000000000000000c:head# 2024-09-18T01:10:58.491 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0891185a:::rbd_data.1176213397f5.000000000000001b:head# 2024-09-18T01:10:58.492 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:08934bd5:::rbd_data.11762ddd5718.000000000000010f:head# 2024-09-18T01:10:58.493 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:08b3d0ad:::rbd_data.11762d95e74a.00000000000000d5:head# 2024-09-18T01:10:58.493 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:08f1aa0a:::rbd_data.1176c2733c33.0000000000000044:head# 2024-09-18T01:10:58.494 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:090a40f8:::rbd_data.11768c71fd0a.0000000000000032:head# 2024-09-18T01:10:58.495 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0985c785:::rbd_data.1176125192ed.0000000000000001:head# 2024-09-18T01:10:58.496 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:09925faa:::rbd_data.117665245fc1.0000000000000093:head# 2024-09-18T01:10:58.496 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:09990027:::rbd_data.1176c2733c33.0000000000000040:head# 2024-09-18T01:10:58.497 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:09a69f94:::rbd_data.1176f0c0f620.0000000000000000:head# 2024-09-18T01:10:58.499 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a54aef7:::rbd_data.11762ddd5718.0000000000000003:head# 2024-09-18T01:10:58.500 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a557d7f:::rbd_data.117677c7714b.0000000000000016:head# 2024-09-18T01:10:58.501 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a5af76d:::rbd_data.11762bc2f01c.0000000000000076:head# 2024-09-18T01:10:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a643317:::rbd_object_map.117668346384:head# 2024-09-18T01:10:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a6c2c56:::rbd_data.11762bc2f01c.000000000000009c:head# 2024-09-18T01:10:58.503 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0a83573d:::rbd_data.1176cffa61f8.0000000000000002:head# 2024-09-18T01:10:58.504 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ac32d34:::rbd_data.11762bc2f01c.000000000000008f:head# 2024-09-18T01:10:58.505 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ae6be27:::rbd_data.117653c0e20f.0000000000000006:head# 2024-09-18T01:10:58.506 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ae9907a:::rbd_object_map.1176b3ffc43:head# 2024-09-18T01:10:58.506 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b450e8e:::rbd_id.image_client.0-clone45:head# 2024-09-18T01:10:58.506 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b548136:::rbd_data.1176cc9d773e.0000000000000003:head# 2024-09-18T01:10:58.507 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head# 2024-09-18T01:10:58.507 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b76c481:::rbd_data.117665245fc1.00000000000000dd:head# 2024-09-18T01:10:58.509 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b7b6f02:::rbd_id.image_client.0-clone30:head# 2024-09-18T01:10:58.509 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0b94c922:::rbd_data.11762ddd5718.000000000000000f:head# 2024-09-18T01:10:58.509 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head# 2024-09-18T01:10:58.512 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0bd0beb7:::rbd_data.1176cffa61f8.000000000000000a:head# 2024-09-18T01:10:58.513 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0c25e909:::rbd_data.11762bc2f01c.000000000000003b:head# 2024-09-18T01:10:58.514 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0c8a3bf8:::rbd_data.11762bc2f01c.0000000000000037:head# 2024-09-18T01:10:58.514 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0cbd23f8:::rbd_data.11762d95e74a.000000000000000b:head# 2024-09-18T01:10:58.516 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0cd26083:::rbd_data.117665245fc1.00000000000000a2:head# 2024-09-18T01:10:58.516 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ceed6d5:::rbd_object_map.1176ec0d676b:head# 2024-09-18T01:10:58.516 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head# 2024-09-18T01:10:58.518 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0d2c989c:::rbd_data.11766701b08b.0000000000000003:head# 2024-09-18T01:10:58.518 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head# 2024-09-18T01:10:58.520 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0d96aea4:::rbd_data.11762bc2f01c.0000000000000079:head# 2024-09-18T01:10:58.520 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:58.519+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:58.521 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0dda50d9:::rbd_data.11762bc2f01c.0000000000000017:head# 2024-09-18T01:10:58.521 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0df4715d:::rbd_data.1176492bc9ca.0000000000000014:head# 2024-09-18T01:10:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0e2b3750:::rbd_data.11762bc2f01c.00000000000001f9:head# 2024-09-18T01:10:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0e64854e:::rbd_object_map.1176c16572b4.000000000000005c:head# 2024-09-18T01:10:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ece5644:::rbd_data.11768c71fd0a.000000000000005a:head# 2024-09-18T01:10:58.523 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0ed2e241:::rbd_data.11769efc726f.0000000000000056:head# 2024-09-18T01:10:58.523 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0f3dc879:::rbd_data.11762ddd5718.00000000000000ae:head# 2024-09-18T01:10:58.524 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0f4f2e2d:::rbd_data.11762ddd5718.0000000000000052:head# 2024-09-18T01:10:58.525 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0f59b482:::rbd_id.image_client.0-clone22:head# 2024-09-18T01:10:58.525 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0f5f3013:::rbd_data.117692a25d3b.0000000000000018:head# 2024-09-18T01:10:58.528 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head# 2024-09-18T01:10:58.530 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0fbfdc14:::rbd_data.1176721d850d.0000000000000005:head# 2024-09-18T01:10:58.531 INFO:teuthology.orchestra.run.smithi162.stderr:Read #3:0fc6a7e6:::rbd_data.11762ddd5718.000000000000007e:head# 2024-09-18T01:10:58.531 INFO:teuthology.orchestra.run.smithi162.stderr:Export successful 2024-09-18T01:10:58.555 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.556 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:10:58.621 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:10:58.620+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:10:58.677 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.677 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.687 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:10:58.722 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:58.721+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:58.793 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.793 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:10:58.823 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:58.822+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:58.903 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:10:58.905 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:58.905 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:58.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:10:58.922 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:10:58.922 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:10:58.924 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:58.923+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:58.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:10:58.926 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:10:59.016 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.016 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.025 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:10:59.026 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:59.024+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:59.058 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 3.10 2024-09-18T01:10:59.126 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:59.125+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:59.134 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.134 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.144 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:10:59.227 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:59.226+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:59.287 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.287 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.296 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:10:59.328 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:59.327+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:59.399 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.400 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.408 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:10:59.429 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:10:59.428+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:10:59.512 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.512 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:10:59.529 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:59.529+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:59.606 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:10:59.631 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.631 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.640 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:10:59.650 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:59.649+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:59.731 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:10:59.730+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:10:59.747 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.747 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.756 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:10:59.832 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:10:59.831+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:10:59.859 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.860 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:10:59.933 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:10:59.932+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:10:59.972 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:10:59.972 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:10:59.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:00.034 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:00.033+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:00.133 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.133 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.135 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:00.135+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:00.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:00.236 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:00.235+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:00.254 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.254 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:00.347 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:00.346+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:00.458 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:00.457+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:00.538 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:00.537+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:00.560 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.560 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:00.609 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:00.668 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:00.667+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:00.729 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.729 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:00.780 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:00.779+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:00.841 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:00.840+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:00.882 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:00.882 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:00.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:00.927 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:00.978 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:00.977+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:01.006 INFO:teuthology.orchestra.run.smithi162.stdout: marking collection for removal 2024-09-18T01:11:01.007 INFO:teuthology.orchestra.run.smithi162.stdout:setting '_remove' omap key 2024-09-18T01:11:01.007 INFO:teuthology.orchestra.run.smithi162.stdout:finish_remove_pgs 3.10_head removing 3.10 2024-09-18T01:11:01.013 INFO:teuthology.orchestra.run.smithi162.stdout:Remove successful 2024-09-18T01:11:01.040 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:01.040 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:01.043 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:01.042+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:01.049 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:01.055 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:01.056 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:01.073 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:01.073 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:01.089 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:01.090 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:01.106 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:01.122 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:01.122 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:01.139 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:01.139 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:01.139 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:01.143 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:01.143+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:01.156 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:01.156 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:01.159 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:01.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:01.160 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:01.160 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:01.160 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:01.161 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:01.169 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:01.254 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:01.253+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:01.313 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:01.313 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:01.313 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:01.313 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:01.314 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:01.322 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:01.363 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:01.362+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:01.446 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:01.445+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.803822957999998, 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.803768732999998, 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.803736275000006, 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.803605012000006, 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.803135026000007, 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.801972426000006, 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.801865144000004, 2024-09-18T01:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.508 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.801783485000001, 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:01.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:01.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.515 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.801268097000005, 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.516 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:01.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.800448858999999, 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:01.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:01.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:01.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.227183772999993, 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:01.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:01.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:01.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.060324881, 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:01.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.539 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.032479492999997, 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:01.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:01.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:01.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.032459193999998, 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:01.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.032438931000002, 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.553 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:01.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.032419078000004, 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:01.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.032399921000007, 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:01.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:01.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.648409494999996, 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:01.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:01.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.588450632000004, 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:01.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:01.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.803947186000002, 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:01.585 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:01.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:01.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:01.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:01.590 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:01.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.10.6 2024-09-18T01:11:01.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:01.593 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:01.590+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:01.692 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:01.691+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:01.746 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:01.746 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:01.746 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:01.746 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:01.756 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:01.799 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:01.798+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:01.860 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:01.860 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:01.860 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:01.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:01.894 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:01.893+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:01.995 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:01.994+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350402193999997, 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350382998000001, 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350364636999998, 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350346533, 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350327935999999, 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350304936000001, 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:02.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:02.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:02.044 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350286263999998, 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.045 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:02.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:02.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:02.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:02.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.051 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350266261000002, 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.052 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350248020999999, 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:02.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350229607000003, 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:02.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.350207580000003, 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:02.066 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:02.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:02.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345955801000002, 2024-09-18T01:11:02.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:02.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:02.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345945348999997, 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:02.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345933658, 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:02.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:02.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345920360999997, 2024-09-18T01:11:02.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.34590429, 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:02.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:02.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:02.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:02.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:02.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:02.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:02.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:02.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.104 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.104 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:02.104 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345887632, 2024-09-18T01:11:02.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:02.104 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:02.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.106 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:02.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:02.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:02.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:02.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:02.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:02.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:02.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:02.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:02.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:02.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.114 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:02.115 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345868336999999, 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.116 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:02.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:02.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:02.124 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345840107000001, 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:02.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.345818983000001, 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:02.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:02.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:02.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:02.134 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:02.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:02.136 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:02.135+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:02.237 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:02.236+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:02.244 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:02.245 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:02.245 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:02.245 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:02.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:02.338 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:02.337+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:02.356 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:02.356 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:02.356 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:02.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:02.439 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:02.438+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:02.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 184.745345902, 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:02.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:02.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:02.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:02.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714319380000006, 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:02.482 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:02.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:02.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:02.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:02.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714309802000002, 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:02.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:02.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:02.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714295776, 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.505 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:02.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:02.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:02.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:02.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:02.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714267003000003, 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:02.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:02.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:02.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714239307, 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.528 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:02.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:02.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:02.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714204859000006, 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:02.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:02.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714175013000002, 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:02.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:02.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714152822000003, 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:02.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:02.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714115915999997, 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:02.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714094801000002, 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.559 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:02.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.714064276000002, 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:02.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708584021999997, 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:02.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708530163000006, 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:02.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708452741000002, 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:02.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708422627000004, 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:02.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708372709000002, 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:02.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.597 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708339121999998, 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:02.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:02.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708314587999993, 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:02.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:02.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:02.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.708129306000004, 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:02.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:02.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:02.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.613 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:02.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:02.614 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:02.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:02.616 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:02.614+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:02.628 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:11:02.716 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:02.715+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:02.763 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:02.763 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:02.763 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:02.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:02.817 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:02.816+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:02.876 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:02.876 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:02.876 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:02.876 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:02.876 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:02.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:02.917 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:02.916+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:03.019 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:03.018+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.429825148999996, 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:03.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:03.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428675134000002, 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:03.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428655823, 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:03.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428628076999999, 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.047 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.051 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428608225000005, 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:03.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428588360999996, 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.056 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:03.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428570155000003, 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:03.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.065 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428550107999996, 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428531899999996, 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.071 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.075 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428513788000004, 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.076 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:03.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:03.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428495870000006, 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.428477849999993, 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:03.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:03.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427924894, 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:03.090 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:03.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427906171000004, 2024-09-18T01:11:03.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:03.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427876388000001, 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:03.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427812299999999, 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:03.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427627998999995, 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:03.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:03.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427609845000006, 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:03.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.120 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427581473000004, 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.427559547000001, 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:03.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:03.130 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:03.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:03.132 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:03.131+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:03.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:03.232 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:03.231+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:03.237 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:03.237 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:03.246 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:03.280 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:03.280 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:03.297 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:03.297 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:03.313 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:03.313 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:03.329 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:03.329 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:03.333 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:03.332+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:03.346 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:03.346 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:03.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:03.363 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:03.363 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:03.380 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:03.380 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:03.383 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:03.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:03.391 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:03.391 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:03.400 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:03.433 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:03.433+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:03.503 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:03.503 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:03.512 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:03.553 INFO:teuthology.orchestra.run.smithi162.stdout:get_pg_num_history pg_num_history pg_num_history(e217 pg_nums {1={14=1},2={18=8},3={23=32}} deleted_pools ) 2024-09-18T01:11:03.554 INFO:teuthology.orchestra.run.smithi162.stdout:Importing pgid 3.10 2024-09-18T01:11:03.557 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:03.556+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:03.559 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:085faf23:::rbd_data.1176ec0d676b.000000000000001b:head# 2024-09-18T01:11:03.559 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.562 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:086f8398:::rbd_data.1176cffa61f8.000000000000000c:head# 2024-09-18T01:11:03.563 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.564 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0891185a:::rbd_data.1176213397f5.000000000000001b:head# 2024-09-18T01:11:03.564 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.565 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:08934bd5:::rbd_data.11762ddd5718.000000000000010f:head# 2024-09-18T01:11:03.565 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.566 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:08b3d0ad:::rbd_data.11762d95e74a.00000000000000d5:head# 2024-09-18T01:11:03.566 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.566 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:08f1aa0a:::rbd_data.1176c2733c33.0000000000000044:head# 2024-09-18T01:11:03.567 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.567 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:090a40f8:::rbd_data.11768c71fd0a.0000000000000032:head# 2024-09-18T01:11:03.567 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.567 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0985c785:::rbd_data.1176125192ed.0000000000000001:head# 2024-09-18T01:11:03.567 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.569 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:09925faa:::rbd_data.117665245fc1.0000000000000093:head# 2024-09-18T01:11:03.569 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.569 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:09990027:::rbd_data.1176c2733c33.0000000000000040:head# 2024-09-18T01:11:03.569 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.571 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:09a69f94:::rbd_data.1176f0c0f620.0000000000000000:head# 2024-09-18T01:11:03.571 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.572 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a54aef7:::rbd_data.11762ddd5718.0000000000000003:head# 2024-09-18T01:11:03.572 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.573 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a557d7f:::rbd_data.117677c7714b.0000000000000016:head# 2024-09-18T01:11:03.573 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.574 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a5af76d:::rbd_data.11762bc2f01c.0000000000000076:head# 2024-09-18T01:11:03.574 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.577 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a643317:::rbd_object_map.117668346384:head# 2024-09-18T01:11:03.577 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.578 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a6c2c56:::rbd_data.11762bc2f01c.000000000000009c:head# 2024-09-18T01:11:03.578 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.578 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0a83573d:::rbd_data.1176cffa61f8.0000000000000002:head# 2024-09-18T01:11:03.578 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.579 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ac32d34:::rbd_data.11762bc2f01c.000000000000008f:head# 2024-09-18T01:11:03.579 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.580 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ae6be27:::rbd_data.117653c0e20f.0000000000000006:head# 2024-09-18T01:11:03.580 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.581 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ae9907a:::rbd_object_map.1176b3ffc43:head# 2024-09-18T01:11:03.581 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.581 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b450e8e:::rbd_id.image_client.0-clone45:head# 2024-09-18T01:11:03.581 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.584 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b548136:::rbd_data.1176cc9d773e.0000000000000003:head# 2024-09-18T01:11:03.584 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.587 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b5785f7:::rbd_data.11769efc726f.0000000000000018:head# 2024-09-18T01:11:03.587 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.589 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b76c481:::rbd_data.117665245fc1.00000000000000dd:head# 2024-09-18T01:11:03.589 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.591 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b7b6f02:::rbd_id.image_client.0-clone30:head# 2024-09-18T01:11:03.591 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.593 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0b94c922:::rbd_data.11762ddd5718.000000000000000f:head# 2024-09-18T01:11:03.593 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.596 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0bca89a9:::rbd_data.11762d3aac1.000000000000000d:head# 2024-09-18T01:11:03.597 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.603 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0bd0beb7:::rbd_data.1176cffa61f8.000000000000000a:head# 2024-09-18T01:11:03.603 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.604 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0c25e909:::rbd_data.11762bc2f01c.000000000000003b:head# 2024-09-18T01:11:03.604 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.605 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0c8a3bf8:::rbd_data.11762bc2f01c.0000000000000037:head# 2024-09-18T01:11:03.605 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.606 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0cbd23f8:::rbd_data.11762d95e74a.000000000000000b:head# 2024-09-18T01:11:03.606 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.607 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0cd26083:::rbd_data.117665245fc1.00000000000000a2:head# 2024-09-18T01:11:03.607 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.608 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ceed6d5:::rbd_object_map.1176ec0d676b:head# 2024-09-18T01:11:03.608 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.610 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head# 2024-09-18T01:11:03.611 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.612 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0d2c989c:::rbd_data.11766701b08b.0000000000000003:head# 2024-09-18T01:11:03.612 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.613 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0d4b2589:::rbd_data.11762afc950e.0000000000000000:head# 2024-09-18T01:11:03.613 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.614 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0d96aea4:::rbd_data.11762bc2f01c.0000000000000079:head# 2024-09-18T01:11:03.614 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.614 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0dda50d9:::rbd_data.11762bc2f01c.0000000000000017:head# 2024-09-18T01:11:03.614 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.616 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:03.616 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:03.617 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0df4715d:::rbd_data.1176492bc9ca.0000000000000014:head# 2024-09-18T01:11:03.617 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.618 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0e2b3750:::rbd_data.11762bc2f01c.00000000000001f9:head# 2024-09-18T01:11:03.618 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.619 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0e64854e:::rbd_object_map.1176c16572b4.000000000000005c:head# 2024-09-18T01:11:03.619 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.619 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ece5644:::rbd_data.11768c71fd0a.000000000000005a:head# 2024-09-18T01:11:03.619 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.620 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0ed2e241:::rbd_data.11769efc726f.0000000000000056:head# 2024-09-18T01:11:03.620 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.623 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0f3dc879:::rbd_data.11762ddd5718.00000000000000ae:head# 2024-09-18T01:11:03.623 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.623 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0f4f2e2d:::rbd_data.11762ddd5718.0000000000000052:head# 2024-09-18T01:11:03.623 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.625 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:03.626 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0f59b482:::rbd_id.image_client.0-clone22:head# 2024-09-18T01:11:03.626 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.626 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0f5f3013:::rbd_data.117692a25d3b.0000000000000018:head# 2024-09-18T01:11:03.626 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.630 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0f6e01e5:::rbd_data.1176ec0d676b.000000000000000d:head# 2024-09-18T01:11:03.631 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.635 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0fbfdc14:::rbd_data.1176721d850d.0000000000000005:head# 2024-09-18T01:11:03.635 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.636 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:03.634+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:03.636 INFO:teuthology.orchestra.run.smithi162.stdout:Write #3:0fc6a7e6:::rbd_data.11762ddd5718.000000000000007e:head# 2024-09-18T01:11:03.636 INFO:teuthology.orchestra.run.smithi162.stdout:snapset 0=[]:{} 2024-09-18T01:11:03.636 INFO:teuthology.orchestra.run.smithi162.stdout:write_pg epoch 217 info 3.10( v 214'485 (0'0,214'485] local-lis/les=82/83 n=50 ec=23/23 lis/c=82/82 les/c/f=83/83/0 sis=82) 2024-09-18T01:11:03.639 INFO:teuthology.orchestra.run.smithi162.stdout:Import successful 2024-09-18T01:11:03.645 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:11:03.736 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:03.735+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:03.774 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:03.774 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:03.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:03.869 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:03.868+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:03.928 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:03.928 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:03.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:03.938 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:03.936+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:04.038 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:04.037+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:04.080 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.080 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.089 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:04.156 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /var/log/ceph/exp.3.10.6 2024-09-18T01:11:04.174 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:04.173+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:04.231 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.232 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.240 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:04.266 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:04.265+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:04.341 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:04.340+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:04.382 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.382 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.391 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:04.441 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:04.440+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:04.493 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.494 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:04.543 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:04.542+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:04.606 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.606 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:04.643 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:04.642+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:04.727 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.727 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:04.744 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:04.743+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:04.843 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.843 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.846 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:04.845+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:04.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:04.946 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:04.946+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:04.953 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:04.953 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:04.962 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:05.048 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:05.047+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:05.065 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:05.065 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:05.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:05.149 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:05.148+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:05.179 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:05.179 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:05.188 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:05.272 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:05.271+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:05.332 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:05.332 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:05.341 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:05.382 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:05.381+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:05.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:05.452 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:05.451+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:05.485 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:05.486 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:05.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:05.504 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:05.504 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:05.520 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:05.521 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:05.536 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:05.536 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:05.552 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:05.552 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:05.551+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:05.568 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:05.569 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:05.585 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:05.585 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:05.585 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:05.602 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:05.606 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:05.607 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:05.639 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:05.640 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:05.649 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:05.653 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:05.652+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:05.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:11:05.754 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:05.753+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:05.762 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:05.762 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:05.762 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:05.762 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:05.762 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:05.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:05.855 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:05.854+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.276737507999997, 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:05.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:05.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275587493000003, 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:05.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275568182000001, 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275540436, 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:05.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275520584000006, 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:05.896 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:05.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:05.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275500719999997, 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.901 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:05.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:05.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275482514000004, 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:05.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:05.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275462466999997, 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.911 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:05.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.915 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275444258999997, 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.920 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275426147000005, 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:05.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:05.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275408229000007, 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:05.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:05.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.275390208999994, 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:05.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274837253000001, 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274818530000005, 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:05.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274788747000002, 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:05.947 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:05.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:05.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:05.952 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274724659, 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:05.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274540357999996, 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:05.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:05.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274522203999993, 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.963 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274493832000005, 2024-09-18T01:11:05.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:05.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:05.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.274471906000002, 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:05.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:05.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:05.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:05.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:05.979 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:05.979 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:05.980 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:05.980+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:06.081 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:06.080+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:06.083 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:06.084 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:06.084 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:06.084 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:06.092 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:06.182 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:06.181+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:06.196 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:06.196 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:06.205 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:06.289 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:06.288+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 234.452316188, 2024-09-18T01:11:06.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:06.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:06.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:06.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.72838781799999, 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:11:06.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.365 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.728365637, 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:06.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 96.760861169999998, 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:06.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:06.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:06.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.378 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.755888841000001, 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:06.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:06.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.750996458000003, 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:06.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:06.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.745581819999998, 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.734007554000002, 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:06.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:06.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:06.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.401 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.725039860999999, 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.402 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:06.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.724985399000005, 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:06.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.719714025000002, 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:06.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:06.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:06.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.719633286000004, 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:06.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:06.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:06.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:06.427 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.718672287000004, 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.428 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:06.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.718613495, 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:06.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:06.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:06.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.439 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.717449526999999, 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:06.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:06.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:06.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.445 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.717084858000007, 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:06.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:06.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.716562322000001, 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:06.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.716164941000002, 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:06.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:06.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.714312211000006, 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:06.464 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:06.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.714190020000004, 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:06.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:06.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:06.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:06.476 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:06.478 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:06.476+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:06.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:06.578 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:06.577+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:06.583 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:06.584 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:06.584 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:06.584 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:06.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:11:06.679 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:06.678+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:06.703 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:06.704 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:06.704 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:06.704 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:06.704 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:06.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:06.798 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:06.797+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:06.865 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:06.865 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:06.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:06.865 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:06.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 189.140555057, 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.866 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:06.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:06.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.872 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.117096966999995, 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.104018714999995, 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103998458000007, 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:06.883 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:06.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103653422999997, 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.889 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:06.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:06.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103346293000001, 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:06.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:06.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103310534000002, 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:06.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:06.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:06.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103272265000001, 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:06.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103247792999994, 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:06.910 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:06.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103227132000001, 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:06.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103208507999994, 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:06.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103188716999995, 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:06.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:06.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103169613999995, 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.932 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:06.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.937 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103151523999998, 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:06.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:06.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:06.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103133424999996, 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:06.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:06.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.948 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103115095999996, 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:06.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:06.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103096730000004, 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:06.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:06.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:06.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:06.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.10307847, 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:06.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:06.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103060016000001, 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.103010330000004, 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.970 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:06.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:06.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:06.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:06.975 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:06.976 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:06.977 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:06.975+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:07.076 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:07.075+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:07.121 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:07.121 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:07.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:07.214 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:07.213+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:07.273 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:07.273 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:07.273 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:07.278 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:07.277+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:07.282 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:07.378 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:07.378+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:07.425 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:07.425 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.511829320000004, 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.426 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:07.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:07.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.511621289000004, 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:07.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:07.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497584713999998, 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.435 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:07.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:07.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497561365999999, 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:07.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.444 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497539845000006, 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:07.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497519126, 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:07.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.453 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497498350000001, 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.454 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:07.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497478204999993, 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:07.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497457828999998, 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:07.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:07.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:07.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:07.467 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497437101000003, 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:07.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497412643000004, 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:07.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:07.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.497392165999997, 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.477 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:07.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488615969999998, 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:07.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488590970000004, 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:07.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488553815000003, 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:07.490 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:07.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488527132000002, 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:07.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488507233000007, 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:07.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:07.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488488025999999, 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:07.503 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:07.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488469013, 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:07.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.488450740000005, 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:07.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:07.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:07.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:07.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:07.516 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:07.517 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:07.518 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:07.516+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:07.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:07.652 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:07.651+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:07.664 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:07.664 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:07.664 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:07.664 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:07.677 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:07.719 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:07.718+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:07.740 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:07.741 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:07.764 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:07.783 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:07.803 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:07.804 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:07.820 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:07.819+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:07.820 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:07.820 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:07.833 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:07.833 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:07.837 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:07.837 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:07.837 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:07.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:07.854 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:07.857 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:07.857 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:07.921 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:07.920+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:07.957 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:07.957 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:07.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:08.022 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:08.021+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:08.108 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.108 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:08.122 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:08.121+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:08.218 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.218 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.223 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:08.222+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:08.227 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:08.323 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:08.322+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:08.370 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.370 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:08.424 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:08.423+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:08.494 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.495 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:08.525 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:08.524+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:08.626 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:08.625+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:08.649 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.649 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.658 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:08.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:11:08.726 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:08.725+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:08.811 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.811 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:08.864 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:08.863+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:08.925 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:08.925 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:08.928 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:08.927+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:08.934 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:09.029 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:09.028+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:09.077 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.077 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:09.129 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:09.129+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:09.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:09.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:09.225 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:11:09.225 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:11:09.229 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.230 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.230 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:09.229+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:09.238 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:09.331 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:09.330+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:09.383 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.383 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.392 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:09.432 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:09.431+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:09.496 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.496 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 233, 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:11:08.665175+0000", 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:10:54.570849+0000", 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:10:43.698009+0000", 2024-09-18T01:11:09.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5806080, 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:11:09.502 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:09.503 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:11:09.504 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:09.505 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:09.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:11:09.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:11:09.509 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:09.510 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.511 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:09.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:09.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:09.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 3, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 3, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.75, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2.25, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2.25, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.515 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:09.516 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "233", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 99, 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 233, 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:09.519 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:09.520 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:11:09.521 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.522 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.528 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.529 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.530 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.531 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:11:09.532 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.533 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:11:09.536 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:11:09.537 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.538 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.540 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:09.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.545 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:11:09.548 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:11:09.549 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.551 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.552 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:09.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 0, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 205, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 219, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.558 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.559 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.560 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:09.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "exists" 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:09.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:09.566 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.567 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.568 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.569 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:11:09.570 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.571 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.572 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.573 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 219 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.574 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.3", 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 4, 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:11:09.575 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:11:09.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 78, 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 80, 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.577 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 82, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 84, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 86, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 88, 2024-09-18T01:11:09.578 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 90, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 92, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 94, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 96, 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 98, 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 2 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:09.580 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 78, 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:11:09.581 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:11:09.582 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:11:09.582 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:11:09.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:09.582 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:09.584 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:09.584 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:09.582+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:09.585 INFO:tasks.thrashosds.thrasher:j is {'epoch': 233, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:11:08.665175+0000', 'last_up_change': '2024-09-18T01:10:54.570849+0000', 'last_in_change': '2024-09-18T01:10:43.698009+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 99, 'snap_epoch': 233, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 205, 'down_at': 219, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 219}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 78, 'length': 1}, {'begin': 80, 'length': 1}, {'begin': 82, 'length': 1}, {'begin': 84, 'length': 1}, {'begin': 86, 'length': 1}, {'begin': 88, 'length': 1}, {'begin': 90, 'length': 1}, {'begin': 92, 'length': 1}, {'begin': 94, 'length': 1}, {'begin': 96, 'length': 1}, {'begin': 98, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 78, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:11:09.585 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:11:09.675 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:11:09.684 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:09.683+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:09.737 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.737 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:09.790 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:09.789+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:09.855 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:09.856 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:09.858 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:09.864 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:09.906 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:09.905+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:09.980 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:09.987 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:09.986+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:09.997 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:10.011 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.011 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:10.012 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:10.012 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:10.012 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.013 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:10.014 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:10.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:10.030 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:10.030 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:10.047 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:10.088 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:10.087+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 192.409554805, 2024-09-18T01:11:10.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:10.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:10.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:10.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.386096714999994, 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:10.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.373018462999994, 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:10.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:10.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372998206000005, 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:10.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372653170999996, 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:10.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:10.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372346041, 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:10.161 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:10.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:10.166 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372310282000001, 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:10.171 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372272013, 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:10.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:10.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372247540999993, 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:10.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372226879999999, 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.186 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372208255999993, 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.187 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:10.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372188464999994, 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.192 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:10.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:10.197 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372169361999994, 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:10.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:10.202 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372151271999996, 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:10.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:10.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:10.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372133172999995, 2024-09-18T01:11:10.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.212 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372114843999995, 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:10.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:10.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.217 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372096478000003, 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:10.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:10.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:10.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372078217999999, 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.223 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372059763999999, 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:10.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:10.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.372010078000002, 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:10.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:10.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.238 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.240 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:10.240 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:10.239+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:10.266 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:10.283 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:10.287 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:10.287 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:10.341 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:10.340+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:10.368 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:10.369 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:10.462 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:10.461+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:10.520 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:10.521 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.529 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:10.570 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:10.569+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:10.643 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:10.642+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:10.673 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.759971265999994, 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.759763234999994, 2024-09-18T01:11:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745726660000003, 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745703312000003, 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745681790999996, 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745661072000004, 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745640296000005, 2024-09-18T01:11:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745620150999997, 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745599775000002, 2024-09-18T01:11:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745579047000007, 2024-09-18T01:11:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745554588999994, 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.745534112000001, 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736757916000002, 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736732915999994, 2024-09-18T01:11:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736695760999993, 2024-09-18T01:11:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736669078000006, 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736649178999997, 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736629972000003, 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736610959000004, 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.736592685999995, 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:10.763 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.765 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:10.765 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:10.764+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:10.866 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:10.865+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:10.909 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:10.909 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:10.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:11.002 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:11.001+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:11.061 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:11.061 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:11.067 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:11.066+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:11.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:11.168 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:11.167+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:11.213 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.550860740999994, 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.550806515999994, 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.550774058000002, 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.550642795000002, 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.550172809000003, 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.549010209000002, 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.548902927, 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.548821267999998, 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.548305880000001, 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.547486641999996, 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.974221556000003, 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.807362663999996, 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.779517276000007, 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.779496976999994, 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.779476713999998, 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.779456861, 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.779437704000003, 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.395447278000006, 2024-09-18T01:11:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.335488415, 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.550984968999998, 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.338 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:11.340 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:11.340 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:11.339+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:11.440 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:11.439+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:11.484 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:11.484 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:11.493 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:11.541 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:11.540+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:11.597 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:11.598 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:11.606 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:11.642 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:11.641+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:11.692 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:11:11.743 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:11.742+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:11.759 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097138595999994, 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097119399999997, 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097101038999995, 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097082935000003, 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097064337999996, 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097041337999997, 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097022666000001, 2024-09-18T01:11:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.097002662999998, 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.096984422999995, 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.096966008999999, 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.096943981999999, 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092692202999999, 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092681751000001, 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092670060000003, 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092656762999994, 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092640692000003, 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092624033999996, 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092604738999995, 2024-09-18T01:11:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092576508999997, 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.092555384999997, 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:11.858 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.859 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.859 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:11.859 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:11.859 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:11.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:11.861 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:11.859+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:11.961 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:11.960+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:11.983 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:11.983 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:11.984 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:11.984 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:11.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:12.062 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:12.061+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:12.117 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:12.118 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:12.118 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:12.118 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:12.118 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:12.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:12.168 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:12.167+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:12.264 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:12.264+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 194.54813755800001, 2024-09-18T01:11:12.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:12.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:12.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.517111036000003, 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:12.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.517101457999999, 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:12.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.517087431999997, 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:12.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:12.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:12.297 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.517058659, 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.517030962999996, 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.303 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:12.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516996515000002, 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:12.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:12.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516966668999999, 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:12.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516944477999999, 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.320 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:12.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:12.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:12.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:12.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516907571999994, 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:12.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516886456999998, 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:12.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.516855931999999, 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.337 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:12.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511375677999993, 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:12.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:12.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:12.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:12.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511321819000003, 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:12.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:12.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.356 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511244396999999, 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:12.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:12.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511214283000001, 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511164364999999, 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.369 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:12.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511130777999995, 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:12.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.511106244000004, 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.380 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:12.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.510920962, 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:12.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:12.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:12.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:12.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:12.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:12.390 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:12.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:12.392 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:12.393 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:12.391+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:12.493 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:12.492+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:12.498 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:12.498 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:12.507 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:12.508 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:12.525 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:12.525 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:12.541 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:12.541 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:12.556 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:12.556 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:12.573 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:12.574 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:12.594 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:12.593+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:12.595 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:12.611 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:12.612 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:12.612 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:12.612 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:12.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:12.616 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:12.621 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:12.690 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:12.695 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:12.694+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:12.729 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:12.759 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:12.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:12.796 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:12.795+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:12.870 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:12.870 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:12.879 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:12.896 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:12.896+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:12.983 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:12.983 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:12.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:12.997 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:12.996+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:13.096 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.096 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.097 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:13.096+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:13.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:13.198 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:13.197+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:13.209 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.209 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.218 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:13.302 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:13.301+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:13.361 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.361 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.370 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:13.412 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:13.411+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:13.500 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:13.499+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:13.515 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.515 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.524 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:13.602 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:13.601+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:13.666 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.666 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.675 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:13.702 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:13.701+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:13.794 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.794 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.803 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:13.801+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:13.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:13.904 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:13.903+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:13.908 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:13.908 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:13.917 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:14.005 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:14.004+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:14.061 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.061 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:14.106 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:14.105+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:14.172 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.172 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:14.206 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:14.205+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:14.284 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.284 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.292 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:14.307 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:14.306+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:14.394 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.394 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.403 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:14.408 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:14.407+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:14.504 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.505 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.509 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:14.508+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:14.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:14.585 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:14.586 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:14.586 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-18T01:11:14.586 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:11:14.586 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:11:14.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-18T01:11:14.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:14.630 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:14.629+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:14.657 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.668 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:14.703 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "" 2024-09-18T01:11:14.704 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:14.707 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:11:14.710 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:14.709+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:14.746 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:14.746 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:14.766 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:14.766 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:14.786 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:14.786 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:14.786 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:14.786 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:14.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:14.805 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:14.811 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:14.810+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:14.831 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:14.855 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:14.855 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:14.856 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:14.876 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:14.879 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:14.880 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:14.912 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:14.911+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:14.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:14.913 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:14.913 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:14.913 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:14.913 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:14.922 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:15.013 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:15.012+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:15.068 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 197.34329084300001, 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312264321000001, 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312254742999997, 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312240716999995, 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312211943999998, 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312184247999994, 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.3121498, 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312119953999996, 2024-09-18T01:11:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312097762999997, 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312060857000006, 2024-09-18T01:11:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312039741999996, 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.312009216999996, 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306528963000005, 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306475104, 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306397681999997, 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306367567999999, 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306317649999997, 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306284063000007, 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306259529000002, 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.306074246999998, 2024-09-18T01:11:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.187 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:15.188 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:15.189 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:15.187+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:15.289 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:15.288+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:15.332 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:15.333 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:15.342 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:15.390 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:15.388+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:15.445 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:15.446 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:15.454 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:15.495 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:15.494+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:15.592 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:15.591+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.997032165999997, 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995882151000004, 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995862840000001, 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995835094, 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995815242000006, 2024-09-18T01:11:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995795377999997, 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995777172000004, 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995757124999997, 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995738916999997, 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995720805000005, 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995702886999993, 2024-09-18T01:11:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995684866999994, 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995131911000001, 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995113188000005, 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995083405000003, 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.995019317000001, 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.994835015999996, 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.994816861999993, 2024-09-18T01:11:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.994788490000005, 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.994766564000003, 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:15.692 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:15.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:15.694 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:15.693+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:15.716 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:11:15.794 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:15.793+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:15.851 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:15.851 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:15.861 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:15.895 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:15.895+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:15.996 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:15.995+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:16.004 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:16.005 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:16.013 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:16.097 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:16.097+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:16.118 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:16.118 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 244.218916046, 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:16.119 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65822 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:34 [read 507904~2048 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.49498767599999, 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057519822999999998, 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65822 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621737+0000", 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.08e-07 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621740+0000", 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621747+0000", 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5279999999999998e-06 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621753+0000", 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679135+0000", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057382678999999999 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679191+0000", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5424000000000003e-05 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679257+0000", 2024-09-18T01:11:16.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6093999999999994e-05 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.494965495, 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:16.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 106.527461028, 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:16.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:16.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:16.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.522488698999993, 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:16.142 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:16.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:16.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.517596315999995, 2024-09-18T01:11:16.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:16.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:16.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:16.152 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.512181678000005, 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:16.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:16.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.500607411999994, 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:16.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:16.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:16.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:16.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.491639719000005, 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.167 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:16.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:16.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.491585256999997, 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:16.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:16.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:16.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.486313882999994, 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.179 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:16.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:16.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.486233143999996, 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:16.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:16.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:16.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.485272144999996, 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:16.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.194 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.485213353000006, 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:16.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.484049385000006, 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:16.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:16.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:16.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.483684715999999, 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.483162179999994, 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:16.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:16.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:16.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.482764798999995, 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:16.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.480912068999999, 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:16.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:16.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:16.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.480789877999996, 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:16.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:16.232 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:16.234 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:16.235 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:16.233+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:16.334 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:16.333+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:16.347 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:16.347 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:16.347 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:16.347 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:16.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:16.440 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:16.439+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:16.508 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:16.535 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:16.534+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:16.637 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:16.636+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:16.653 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 198.92869419600001, 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.905236106000004, 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.892157854000004, 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.892137597000001, 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891792562000006, 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891485431999996, 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891449672999997, 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891411403999996, 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891386932000003, 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891366270999995, 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891347647000003, 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891327856000004, 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891308753000004, 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891290663000007, 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891272564000005, 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891254235000005, 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891235868999999, 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891217608999995, 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891199154999995, 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.891149468999998, 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:16.764 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:16.765 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:16.766 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:16.764+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:16.866 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:16.865+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:16.877 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:16.878 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:16.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:16.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:16.967 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:16.967+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:17.017 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:17.044 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:17.044 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:17.050 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.050 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.060 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:17.061 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:17.061 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:17.077 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:17.077 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:17.077 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:17.076+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:17.097 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:17.097 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:17.131 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:17.131 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:17.135 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:17.135 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:17.167 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.167 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.168 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:17.167+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:17.176 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:17.269 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:17.268+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:17.319 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.319 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.328 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:17.372 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:17.371+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:17.431 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.431 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:17.481 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:17.480+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:17.571 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:17.570+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:17.676 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:17.675+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:17.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:11:17.741 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.741 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.751 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:17.793 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:17.792+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:17.873 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:17.872+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:17.905 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:17.905 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:17.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:17.973 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:17.972+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.068 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:18.074 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:18.073+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:18.174 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:18.173+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:18.182 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.182 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.192 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:18.276 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:18.275+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:18.335 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.336 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.344 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:18.376 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:18.375+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:18.477 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:18.476+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:18.488 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.488 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:18.578 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:18.577+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:18.642 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.642 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:18.679 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:18.678+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:18.725 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:11:18.758 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.758 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.767 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:18.812 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:18.811+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:18.881 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:18.880+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:18.882 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:18.882 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:18.891 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:18.983 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:18.982+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:19.035 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:19.035 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:19.044 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:19.088 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:19.084+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:19.136 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:19.185 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:19.184+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:19.188 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:19.188 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:19.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:19.256 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:19.272 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:19.273 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:19.286 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:19.285+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:19.288 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:19.301 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:19.301 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:19.304 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:19.310 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:19.321 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:19.321 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:19.338 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:19.338 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:19.338 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:19.355 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:19.355 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:19.359 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:19.359 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:19.401 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:19.400+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:19.415 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:19.415 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:19.415 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:19.415 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:19.424 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:19.488 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:19.487+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:19.567 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:19.567 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:19.567 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:19.575 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:19.619 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:19.618+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:19.680 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:19.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 201.95566135000001, 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.681 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:19.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:19.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.932203259999994, 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.690 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:19.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:19.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.919125007999995, 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:19.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:19.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:19.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.919104751000006, 2024-09-18T01:11:19.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:19.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918759715999997, 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.707 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918452586000001, 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:19.712 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:19.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:19.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918416827000001, 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:19.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:19.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:19.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918378558000001, 2024-09-18T01:11:19.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:19.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:19.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918354085999994, 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:19.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:19.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918333425, 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.735 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:19.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:19.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:19.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:19.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918314800999994, 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:19.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:19.746 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918295009999994, 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:19.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:19.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918275906999995, 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.752 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:19.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:19.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918257816999997, 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:19.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918239717999995, 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.764 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:19.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:19.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918221388999996, 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:19.769 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:19.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:19.775 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918203023000004, 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:19.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918184762999999, 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:19.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:19.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918166309, 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:19.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:19.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.918116623000003, 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.794 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:19.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:19.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:19.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:19.799 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:19.799 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:19.800 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:11:19.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:11:19.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:19.802 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:19.799+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:19.901 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:19.900+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:19.947 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:19.947 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:19.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:20.002 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:20.001+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:20.104 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:20.105 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:20.105 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:20.104+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:20.111 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 244, 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:11:19.733691+0000", 2024-09-18T01:11:20.130 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:10:54.570849+0000", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:10:43.698009+0000", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5806080, 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.131 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:11:20.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:20.133 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:20.134 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:20.135 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:20.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:20.137 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:11:20.138 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:20.139 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:11:20.140 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:20.141 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:20.142 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.144 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 3, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 3, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.75, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2.25, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2.25, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.145 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:20.146 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:20.147 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "244", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 99, 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 244, 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:20.148 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:20.149 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:20.150 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:11:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.152 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.154 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:11:20.157 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.158 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:11:20.159 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.160 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:11:20.163 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.164 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.165 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.167 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.168 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:20.170 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.173 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.174 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:11:20.176 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.179 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.180 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.181 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.182 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:11:20.183 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.184 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.186 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.187 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 0, 2024-09-18T01:11:20.188 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 205, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 219, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.189 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.192 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:11:20.193 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "exists" 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:20.194 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:11:20.198 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:20.199 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.200 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:11:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.201 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.202 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.203 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.205 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 219 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:20.206 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.3", 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 4, 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:11:20.208 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 56, 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.209 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 58, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 60, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 62, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 64, 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 66, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 68, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 70, 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 72, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 74, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 76, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 78, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 80, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 82, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.213 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 84, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 86, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 88, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 90, 2024-09-18T01:11:20.214 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 92, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 94, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 96, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 98, 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 2 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 56, 2024-09-18T01:11:20.216 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:11:20.217 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:11:20.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.218 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:20.220 INFO:tasks.thrashosds.thrasher:j is {'epoch': 244, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:11:19.733691+0000', 'last_up_change': '2024-09-18T01:10:54.570849+0000', 'last_in_change': '2024-09-18T01:10:43.698009+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '244', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 99, 'snap_epoch': 244, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 205, 'down_at': 219, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 219}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 56, 'length': 1}, {'begin': 58, 'length': 1}, {'begin': 60, 'length': 1}, {'begin': 62, 'length': 1}, {'begin': 64, 'length': 1}, {'begin': 66, 'length': 1}, {'begin': 68, 'length': 1}, {'begin': 70, 'length': 1}, {'begin': 72, 'length': 1}, {'begin': 74, 'length': 1}, {'begin': 76, 'length': 1}, {'begin': 78, 'length': 1}, {'begin': 80, 'length': 1}, {'begin': 82, 'length': 1}, {'begin': 84, 'length': 1}, {'begin': 86, 'length': 1}, {'begin': 88, 'length': 1}, {'begin': 90, 'length': 1}, {'begin': 92, 'length': 1}, {'begin': 94, 'length': 1}, {'begin': 96, 'length': 1}, {'begin': 98, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 56, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:11:20.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:11:20.221 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:20.219+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:20.320 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:20.319+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:20.324 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.324 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:20.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.410773949000003, 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.325 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.410565918000003, 2024-09-18T01:11:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396529342999997, 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396505994999998, 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:20.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396484474000005, 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:20.343 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:20.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396463754999999, 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.348 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396442979, 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:20.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396422834000006, 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:20.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:20.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.361 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396402457999997, 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:20.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.366 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396381730000002, 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:20.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.371 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396357272000003, 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:20.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.396336794999996, 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:20.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387560598999997, 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.381 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:20.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.385 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387535599000003, 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.386 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:20.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387498444000002, 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.391 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:20.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387471761, 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.395 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:20.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387451862000006, 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:20.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387432654999998, 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:20.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387413641999999, 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.408 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:20.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.387395369000004, 2024-09-18T01:11:20.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:20.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.416 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:20.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:20.459 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:20.458+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:20.521 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:20.521+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:11:20.543 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:11:20.555 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-18T01:11:19.739302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1033810378,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":259614,"num_read_kb":1257219,"num_write":34464,"num_write_kb":851998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2256,"num_bytes_recovered":275813679,"num_keys_recovered":242,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23800,"ondisk_log_size":23800,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1021796,"kb_used_data":539028,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":753952924,"statfs":{"total":773094113280,"available":772047794176,"internally_reserved":0,"allocated":551964672,"data_stored":536928973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-180463361,"num_objects":-288,"num_object_clones":0,"num_object_copies":-576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-288,"num_whiteouts":0,"num_read":1117,"num_read_kb":963,"num_write":460,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025627"},"pg_stats":[{"pgid":"3.1f","version":"240'499","reported_seq":7045,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.727934+0000","last_change":"2024-09-18T01:11:17.728117+0000","last_active":"2024-09-18T01:11:17.727934+0000","last_peered":"2024-09-18T01:11:17.727934+0000","last_clean":"2024-09-18T01:11:17.727934+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:11:17.727934+0000","last_undegraded":"2024-09-18T01:11:17.727934+0000","last_fullsized":"2024-09-18T01:11:17.727934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:05.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000808137,"stat_sum":{"num_bytes":31219468,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6275,"num_read_kb":39533,"num_write":606,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1e","version":"239'705","reported_seq":4195,"reported_epoch":239,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:14.843751+0000","last_change":"2024-09-18T01:10:55.580235+0000","last_active":"2024-09-18T01:11:14.843751+0000","last_peered":"2024-09-18T01:11:14.843751+0000","last_clean":"2024-09-18T01:10:53.747252+0000","last_became_active":"2024-09-18T01:10:55.580235+0000","last_became_peered":"2024-09-18T01:10:55.580235+0000","last_unstale":"2024-09-18T01:11:14.843751+0000","last_undegraded":"2024-09-18T01:10:55.578208+0000","last_fullsized":"2024-09-18T01:10:55.578105+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":18,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:04:54.403305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058770700000000003,"stat_sum":{"num_bytes":11883779,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":4776,"num_read_kb":33843,"num_write":1101,"num_write_kb":25798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":25}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"240'615","reported_seq":4507,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.735864+0000","last_change":"2024-09-18T01:11:17.736219+0000","last_active":"2024-09-18T01:11:17.735864+0000","last_peered":"2024-09-18T01:11:17.735864+0000","last_clean":"2024-09-18T01:11:17.735864+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:11:17.735864+0000","last_undegraded":"2024-09-18T01:11:17.735864+0000","last_fullsized":"2024-09-18T01:11:17.735864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:00.462850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570831,"stat_sum":{"num_bytes":31111320,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":4361,"num_read_kb":36504,"num_write":940,"num_write_kb":39198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1c","version":"242'603","reported_seq":5618,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.755363+0000","last_change":"2024-09-18T01:11:17.735695+0000","last_active":"2024-09-18T01:11:17.755363+0000","last_peered":"2024-09-18T01:11:17.755363+0000","last_clean":"2024-09-18T01:11:17.755363+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:11:17.755363+0000","last_undegraded":"2024-09-18T01:11:17.755363+0000","last_fullsized":"2024-09-18T01:11:17.755363+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:19.706867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014623209999999999,"stat_sum":{"num_bytes":26683431,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5177,"num_read_kb":38986,"num_write":816,"num_write_kb":35495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1b","version":"240'1152","reported_seq":7093,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.727036+0000","last_change":"2024-09-18T01:11:17.727332+0000","last_active":"2024-09-18T01:11:17.727036+0000","last_peered":"2024-09-18T01:11:17.727036+0000","last_clean":"2024-09-18T01:11:17.727036+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:11:17.727036+0000","last_undegraded":"2024-09-18T01:11:17.727036+0000","last_fullsized":"2024-09-18T01:11:17.727036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1152,"log_dups_size":0,"ondisk_log_size":1152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:59.560202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052041999999999995,"stat_sum":{"num_bytes":25246604,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":9509,"num_read_kb":42031,"num_write":1702,"num_write_kb":30410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1a","version":"239'465","reported_seq":7238,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-18T01:11:15.736680+0000","last_change":"2024-09-18T01:11:15.715423+0000","last_active":"2024-09-18T01:11:15.736680+0000","last_peered":"2024-09-18T01:11:15.736680+0000","last_clean":"2024-09-18T01:11:15.736680+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:11:15.736680+0000","last_undegraded":"2024-09-18T01:11:15.736680+0000","last_fullsized":"2024-09-18T01:11:15.736680+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:36:05.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042842200000000003,"stat_sum":{"num_bytes":17273532,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":6224,"num_read_kb":37918,"num_write":602,"num_write_kb":25987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.19","version":"241'662","reported_seq":9338,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.725860+0000","last_change":"2024-09-18T01:11:17.726094+0000","last_active":"2024-09-18T01:11:17.725860+0000","last_peered":"2024-09-18T01:11:17.725860+0000","last_clean":"2024-09-18T01:11:17.725860+0000","last_became_active":"2024-09-18T01:07:28.971584+0000","last_became_peered":"2024-09-18T01:07:28.971584+0000","last_unstale":"2024-09-18T01:11:17.725860+0000","last_undegraded":"2024-09-18T01:11:17.725860+0000","last_fullsized":"2024-09-18T01:11:17.725860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:19.849430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041227400000000002,"stat_sum":{"num_bytes":36587386,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":8977,"num_read_kb":43084,"num_write":756,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.18","version":"239'526","reported_seq":13116,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-18T01:11:14.827095+0000","last_change":"2024-09-18T01:11:14.776429+0000","last_active":"2024-09-18T01:11:14.827095+0000","last_peered":"2024-09-18T01:11:14.827095+0000","last_clean":"2024-09-18T01:11:14.827095+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:11:14.827095+0000","last_undegraded":"2024-09-18T01:11:14.827095+0000","last_fullsized":"2024-09-18T01:11:14.827095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:58.741848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000681574,"stat_sum":{"num_bytes":47285161,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":12776,"num_read_kb":57677,"num_write":634,"num_write_kb":24315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.17","version":"241'1176","reported_seq":7522,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.727994+0000","last_change":"2024-09-18T01:11:17.728173+0000","last_active":"2024-09-18T01:11:17.727994+0000","last_peered":"2024-09-18T01:11:17.727994+0000","last_clean":"2024-09-18T01:11:17.727994+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:11:17.727994+0000","last_undegraded":"2024-09-18T01:11:17.727994+0000","last_fullsized":"2024-09-18T01:11:17.727994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1176,"log_dups_size":0,"ondisk_log_size":1176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:18:44.281532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395261,"stat_sum":{"num_bytes":30636262,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":9090,"num_read_kb":41630,"num_write":1904,"num_write_kb":28910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.16","version":"240'675","reported_seq":4465,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.735475+0000","last_change":"2024-09-18T01:11:17.735770+0000","last_active":"2024-09-18T01:11:17.735475+0000","last_peered":"2024-09-18T01:11:17.735475+0000","last_clean":"2024-09-18T01:11:17.735475+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:11:17.735475+0000","last_undegraded":"2024-09-18T01:11:17.735475+0000","last_fullsized":"2024-09-18T01:11:17.735475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:24.387407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475255,"stat_sum":{"num_bytes":26422918,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4566,"num_read_kb":22143,"num_write":939,"num_write_kb":19421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.15","version":"240'878","reported_seq":9602,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.725086+0000","last_change":"2024-09-18T01:11:16.725341+0000","last_active":"2024-09-18T01:11:16.725086+0000","last_peered":"2024-09-18T01:11:16.725086+0000","last_clean":"2024-09-18T01:11:16.725086+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:11:16.725086+0000","last_undegraded":"2024-09-18T01:11:16.725086+0000","last_fullsized":"2024-09-18T01:11:16.725086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:07:04.912142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092865799999999996,"stat_sum":{"num_bytes":38823989,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":10548,"num_read_kb":60311,"num_write":1259,"num_write_kb":30037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.14","version":"239'624","reported_seq":6184,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-18T01:11:15.715440+0000","last_change":"2024-09-18T01:11:15.715599+0000","last_active":"2024-09-18T01:11:15.715440+0000","last_peered":"2024-09-18T01:11:15.715440+0000","last_clean":"2024-09-18T01:11:15.715440+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:11:15.715440+0000","last_undegraded":"2024-09-18T01:11:15.715440+0000","last_fullsized":"2024-09-18T01:11:15.715440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:33:54.467890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046656200000000001,"stat_sum":{"num_bytes":42378699,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6363,"num_read_kb":23424,"num_write":926,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":305,"reported_epoch":241,"state":"active+undersized","last_fresh":"2024-09-18T01:11:16.724068+0000","last_change":"2024-09-18T01:10:55.580802+0000","last_active":"2024-09-18T01:11:16.724068+0000","last_peered":"2024-09-18T01:11:16.724068+0000","last_clean":"2024-09-18T01:09:55.569213+0000","last_became_active":"2024-09-18T01:10:55.580802+0000","last_became_peered":"2024-09-18T01:10:55.580802+0000","last_unstale":"2024-09-18T01:11:16.724068+0000","last_undegraded":"2024-09-18T01:11:16.724068+0000","last_fullsized":"2024-09-18T01:10:55.579666+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:34.355623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"241'674","reported_seq":4397,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726925+0000","last_change":"2024-09-18T01:11:17.727145+0000","last_active":"2024-09-18T01:11:17.726925+0000","last_peered":"2024-09-18T01:11:17.726925+0000","last_clean":"2024-09-18T01:11:17.726925+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:11:17.726925+0000","last_undegraded":"2024-09-18T01:11:17.726925+0000","last_fullsized":"2024-09-18T01:11:17.726925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.422525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127000000000003,"stat_sum":{"num_bytes":10354584,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4679,"num_read_kb":27193,"num_write":955,"num_write_kb":22606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":464,"reported_epoch":242,"state":"active+undersized","last_fresh":"2024-09-18T01:11:17.725868+0000","last_change":"2024-09-18T01:10:55.576107+0000","last_active":"2024-09-18T01:11:17.725868+0000","last_peered":"2024-09-18T01:11:17.725868+0000","last_clean":"2024-09-18T01:10:53.742769+0000","last_became_active":"2024-09-18T01:10:55.576107+0000","last_became_peered":"2024-09-18T01:10:55.576107+0000","last_unstale":"2024-09-18T01:11:17.725868+0000","last_undegraded":"2024-09-18T01:11:17.725868+0000","last_fullsized":"2024-09-18T01:10:55.575499+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:34.049490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"239'455","reported_seq":7522,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-18T01:11:14.777898+0000","last_change":"2024-09-18T01:11:14.777898+0000","last_active":"2024-09-18T01:11:14.777898+0000","last_peered":"2024-09-18T01:11:14.777898+0000","last_clean":"2024-09-18T01:11:14.777898+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:11:14.777898+0000","last_undegraded":"2024-09-18T01:11:14.777898+0000","last_fullsized":"2024-09-18T01:11:14.777898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:35.567410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013158480000000001,"stat_sum":{"num_bytes":29061785,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":717,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":459,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.724391+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:11:16.724391+0000","last_peered":"2024-09-18T01:11:16.724391+0000","last_clean":"2024-09-18T01:11:16.724391+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:11:16.724391+0000","last_undegraded":"2024-09-18T01:11:16.724391+0000","last_fullsized":"2024-09-18T01:11:16.724391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"240'528","reported_seq":6894,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726580+0000","last_change":"2024-09-18T01:11:17.726809+0000","last_active":"2024-09-18T01:11:17.726580+0000","last_peered":"2024-09-18T01:11:17.726580+0000","last_clean":"2024-09-18T01:11:17.726580+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:11:17.726580+0000","last_undegraded":"2024-09-18T01:11:17.726580+0000","last_fullsized":"2024-09-18T01:11:17.726580+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:01:59.898357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481022,"stat_sum":{"num_bytes":34389897,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5657,"num_read_kb":52457,"num_write":694,"num_write_kb":28694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":462,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726375+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:11:17.726375+0000","last_peered":"2024-09-18T01:11:17.726375+0000","last_clean":"2024-09-18T01:11:17.726375+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:11:17.726375+0000","last_undegraded":"2024-09-18T01:11:17.726375+0000","last_fullsized":"2024-09-18T01:11:17.726375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"241'906","reported_seq":12576,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.735790+0000","last_change":"2024-09-18T01:11:17.736140+0000","last_active":"2024-09-18T01:11:17.735790+0000","last_peered":"2024-09-18T01:11:17.735790+0000","last_clean":"2024-09-18T01:11:17.735790+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:11:17.735790+0000","last_undegraded":"2024-09-18T01:11:17.735790+0000","last_fullsized":"2024-09-18T01:11:17.735790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.366089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015353809999999999,"stat_sum":{"num_bytes":52813127,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":13936,"num_read_kb":64815,"num_write":1398,"num_write_kb":38726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.3","version":"241'560","reported_seq":8668,"reported_epoch":242,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:17.726337+0000","last_change":"2024-09-18T01:10:55.580459+0000","last_active":"2024-09-18T01:11:17.726337+0000","last_peered":"2024-09-18T01:11:17.726337+0000","last_clean":"2024-09-18T01:10:50.027978+0000","last_became_active":"2024-09-18T01:10:55.580459+0000","last_became_peered":"2024-09-18T01:10:55.580459+0000","last_unstale":"2024-09-18T01:11:17.726337+0000","last_undegraded":"2024-09-18T01:10:55.578156+0000","last_fullsized":"2024-09-18T01:10:55.578070+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":21,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:50.493136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039494299999999998,"stat_sum":{"num_bytes":13351605,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8176,"num_read_kb":43509,"num_write":769,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":21}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":463,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.734235+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:11:17.734235+0000","last_peered":"2024-09-18T01:11:17.734235+0000","last_clean":"2024-09-18T01:11:17.734235+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:11:17.734235+0000","last_undegraded":"2024-09-18T01:11:17.734235+0000","last_fullsized":"2024-09-18T01:11:17.734235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"240'669","reported_seq":13346,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726509+0000","last_change":"2024-09-18T01:11:17.726810+0000","last_active":"2024-09-18T01:11:17.726509+0000","last_peered":"2024-09-18T01:11:17.726509+0000","last_clean":"2024-09-18T01:11:17.726509+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:11:17.726509+0000","last_undegraded":"2024-09-18T01:11:17.726509+0000","last_fullsized":"2024-09-18T01:11:17.726509+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:52:18.975485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001133565,"stat_sum":{"num_bytes":46574535,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":12625,"num_read_kb":60625,"num_write":1032,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.725273+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:11:17.725273+0000","last_peered":"2024-09-18T01:11:17.725273+0000","last_clean":"2024-09-18T01:11:17.725273+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:11:17.725273+0000","last_undegraded":"2024-09-18T01:11:17.725273+0000","last_fullsized":"2024-09-18T01:11:17.725273+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"241'839","reported_seq":15881,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726550+0000","last_change":"2024-09-18T01:11:17.726948+0000","last_active":"2024-09-18T01:11:17.726550+0000","last_peered":"2024-09-18T01:11:17.726550+0000","last_clean":"2024-09-18T01:11:17.726550+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:11:17.726550+0000","last_undegraded":"2024-09-18T01:11:17.726550+0000","last_fullsized":"2024-09-18T01:11:17.726550+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:46.476824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088337300000000001,"stat_sum":{"num_bytes":78038516,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":16725,"num_read_kb":44700,"num_write":985,"num_write_kb":34541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":459,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.723913+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:11:16.723913+0000","last_peered":"2024-09-18T01:11:16.723913+0000","last_clean":"2024-09-18T01:11:16.723913+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:11:16.723913+0000","last_undegraded":"2024-09-18T01:11:16.723913+0000","last_fullsized":"2024-09-18T01:11:16.723913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"239'475","reported_seq":8182,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-18T01:11:14.829789+0000","last_change":"2024-09-18T01:11:14.776393+0000","last_active":"2024-09-18T01:11:14.829789+0000","last_peered":"2024-09-18T01:11:14.829789+0000","last_clean":"2024-09-18T01:11:14.829789+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:11:14.829789+0000","last_undegraded":"2024-09-18T01:11:14.829789+0000","last_fullsized":"2024-09-18T01:11:14.829789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:50:06.432874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053646900000000003,"stat_sum":{"num_bytes":31115662,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7190,"num_read_kb":33149,"num_write":564,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.3","version":"19'1","reported_seq":498,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.734045+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:11:17.734045+0000","last_peered":"2024-09-18T01:11:17.734045+0000","last_clean":"2024-09-18T01:11:17.734045+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:11:17.734045+0000","last_undegraded":"2024-09-18T01:11:17.734045+0000","last_fullsized":"2024-09-18T01:11:17.734045+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":612,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.723924+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:11:16.723924+0000","last_peered":"2024-09-18T01:11:16.723924+0000","last_clean":"2024-09-18T01:11:16.723924+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:11:16.723924+0000","last_undegraded":"2024-09-18T01:11:16.723924+0000","last_fullsized":"2024-09-18T01:11:16.723924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"239'1232","reported_seq":4622,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-18T01:11:14.807097+0000","last_change":"2024-09-18T01:11:14.776865+0000","last_active":"2024-09-18T01:11:14.807097+0000","last_peered":"2024-09-18T01:11:14.807097+0000","last_clean":"2024-09-18T01:11:14.807097+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:11:14.807097+0000","last_undegraded":"2024-09-18T01:11:14.807097+0000","last_fullsized":"2024-09-18T01:11:14.807097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:26.487899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077496599999999998,"stat_sum":{"num_bytes":17649773,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6657,"num_read_kb":21584,"num_write":1976,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.9","version":"239'722","reported_seq":12485,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-18T01:11:15.715207+0000","last_change":"2024-09-18T01:11:15.715386+0000","last_active":"2024-09-18T01:11:15.715207+0000","last_peered":"2024-09-18T01:11:15.715207+0000","last_clean":"2024-09-18T01:11:15.715207+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:11:15.715207+0000","last_undegraded":"2024-09-18T01:11:15.715207+0000","last_fullsized":"2024-09-18T01:11:15.715207+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:32:56.228114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041017000000000001,"stat_sum":{"num_bytes":40871057,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12010,"num_read_kb":43867,"num_write":1022,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.a","version":"239'461","reported_seq":10627,"reported_epoch":240,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:15.715055+0000","last_change":"2024-09-18T01:10:55.576692+0000","last_active":"2024-09-18T01:11:15.715055+0000","last_peered":"2024-09-18T01:11:15.715055+0000","last_clean":"2024-09-18T01:10:50.135462+0000","last_became_active":"2024-09-18T01:10:55.576692+0000","last_became_peered":"2024-09-18T01:10:55.576692+0000","last_unstale":"2024-09-18T01:11:15.715055+0000","last_undegraded":"2024-09-18T01:10:55.576179+0000","last_fullsized":"2024-09-18T01:10:55.576069+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:49.240705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058732200000000004,"stat_sum":{"num_bytes":54251955,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":10118,"num_read_kb":42770,"num_write":591,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":32}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"239'678","reported_seq":4896,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-18T01:11:14.776989+0000","last_change":"2024-09-18T01:11:14.776989+0000","last_active":"2024-09-18T01:11:14.776989+0000","last_peered":"2024-09-18T01:11:14.776989+0000","last_clean":"2024-09-18T01:11:14.776989+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:11:14.776989+0000","last_undegraded":"2024-09-18T01:11:14.776989+0000","last_fullsized":"2024-09-18T01:11:14.776989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:19.023379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075268899999999996,"stat_sum":{"num_bytes":17646782,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4893,"num_read_kb":33497,"num_write":917,"num_write_kb":24400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.c","version":"240'975","reported_seq":6751,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.734384+0000","last_change":"2024-09-18T01:11:17.734609+0000","last_active":"2024-09-18T01:11:17.734384+0000","last_peered":"2024-09-18T01:11:17.734384+0000","last_clean":"2024-09-18T01:11:17.734384+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:11:17.734384+0000","last_undegraded":"2024-09-18T01:11:17.734384+0000","last_fullsized":"2024-09-18T01:11:17.734384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:07:48.350521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398756,"stat_sum":{"num_bytes":13615873,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":7731,"num_read_kb":32509,"num_write":1633,"num_write_kb":22642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.d","version":"240'671","reported_seq":14028,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.724422+0000","last_change":"2024-09-18T01:11:16.724709+0000","last_active":"2024-09-18T01:11:16.724422+0000","last_peered":"2024-09-18T01:11:16.724422+0000","last_clean":"2024-09-18T01:11:16.724422+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:11:16.724422+0000","last_undegraded":"2024-09-18T01:11:16.724422+0000","last_fullsized":"2024-09-18T01:11:16.724422+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:51.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048757,"stat_sum":{"num_bytes":56448980,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":53109,"num_write":925,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.e","version":"241'1243","reported_seq":7666,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.725288+0000","last_change":"2024-09-18T01:11:16.725288+0000","last_active":"2024-09-18T01:11:16.725288+0000","last_peered":"2024-09-18T01:11:16.725288+0000","last_clean":"2024-09-18T01:11:16.725288+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:11:16.725288+0000","last_undegraded":"2024-09-18T01:11:16.725288+0000","last_fullsized":"2024-09-18T01:11:16.725288+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1243,"log_dups_size":0,"ondisk_log_size":1243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:32.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080966399999999998,"stat_sum":{"num_bytes":33370468,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":9324,"num_read_kb":56615,"num_write":1999,"num_write_kb":38810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.f","version":"240'627","reported_seq":5903,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.725200+0000","last_change":"2024-09-18T01:11:16.725439+0000","last_active":"2024-09-18T01:11:16.725200+0000","last_peered":"2024-09-18T01:11:16.725200+0000","last_clean":"2024-09-18T01:11:16.725200+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:11:16.725200+0000","last_undegraded":"2024-09-18T01:11:16.725200+0000","last_fullsized":"2024-09-18T01:11:16.725200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:54:42.832564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001002223,"stat_sum":{"num_bytes":36415301,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5772,"num_read_kb":31053,"num_write":894,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.10","version":"240'518","reported_seq":4200,"reported_epoch":242,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:17.725906+0000","last_change":"2024-09-18T01:10:55.580456+0000","last_active":"2024-09-18T01:11:17.725906+0000","last_peered":"2024-09-18T01:11:17.725906+0000","last_clean":"2024-09-18T01:10:49.782749+0000","last_became_active":"2024-09-18T01:10:55.580456+0000","last_became_peered":"2024-09-18T01:10:55.580456+0000","last_unstale":"2024-09-18T01:11:17.725906+0000","last_undegraded":"2024-09-18T01:10:55.578006+0000","last_fullsized":"2024-09-18T01:10:55.577905+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":21,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:37:46.245896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038286899999999998,"stat_sum":{"num_bytes":12530668,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3820,"num_read_kb":28572,"num_write":696,"num_write_kb":22795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"240'625","reported_seq":4713,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.724779+0000","last_change":"2024-09-18T01:11:16.725015+0000","last_active":"2024-09-18T01:11:16.724779+0000","last_peered":"2024-09-18T01:11:16.724779+0000","last_clean":"2024-09-18T01:11:16.724779+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:11:16.724779+0000","last_undegraded":"2024-09-18T01:11:16.724779+0000","last_fullsized":"2024-09-18T01:11:16.724779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:52.634186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075887600000000002,"stat_sum":{"num_bytes":53045315,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":824,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.12","version":"241'1343","reported_seq":6557,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-18T01:11:17.726618+0000","last_change":"2024-09-18T01:11:17.726859+0000","last_active":"2024-09-18T01:11:17.726618+0000","last_peered":"2024-09-18T01:11:17.726618+0000","last_clean":"2024-09-18T01:11:17.726618+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:11:17.726618+0000","last_undegraded":"2024-09-18T01:11:17.726618+0000","last_fullsized":"2024-09-18T01:11:17.726618+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:27:39.832024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064370599999999997,"stat_sum":{"num_bytes":13862708,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8847,"num_read_kb":29432,"num_write":2234,"num_write_kb":20043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.13","version":"241'899","reported_seq":5930,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-18T01:11:16.724809+0000","last_change":"2024-09-18T01:11:16.724809+0000","last_active":"2024-09-18T01:11:16.724809+0000","last_peered":"2024-09-18T01:11:16.724809+0000","last_clean":"2024-09-18T01:11:16.724809+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:11:16.724809+0000","last_undegraded":"2024-09-18T01:11:16.724809+0000","last_fullsized":"2024-09-18T01:11:16.724809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:21.958407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059664799999999997,"stat_sum":{"num_bytes":22258851,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7138,"num_read_kb":22846,"num_write":1293,"num_write_kb":16539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1033219991,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":259497,"num_read_kb":1257118,"num_write":34303,"num_write_kb":849580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2252,"num_bytes_recovered":275813679,"num_keys_recovered":242,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":527626240,"data_stored":524318970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23680,"ondisk_log_size":23680,"up":60,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":158020,"kb_used_data":72688,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94213820,"statfs":{"total":96636764160,"available":96474951680,"internally_reserved":0,"allocated":74432512,"data_stored":72583149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87357898},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85196,"kb_used_data":45680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286644,"statfs":{"total":96636764160,"available":96549523456,"internally_reserved":0,"allocated":46776320,"data_stored":45024779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":158364,"kb_used_data":82756,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94213476,"statfs":{"total":96636764160,"available":96474599424,"internally_reserved":0,"allocated":84742144,"data_stored":82858610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72668,"kb_used_data":33160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94299172,"statfs":{"total":96636764160,"available":96562352128,"internally_reserved":0,"allocated":33955840,"data_stored":32291572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97024,"kb_used_data":47784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94274816,"statfs":{"total":96636764160,"available":96537411584,"internally_reserved":0,"allocated":48930816,"data_stored":47146059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133400,"kb_used_data":70268,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94238440,"statfs":{"total":96636764160,"available":96500162560,"internally_reserved":0,"allocated":71954432,"data_stored":70062273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166252,"kb_used_data":98948,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94205588,"statfs":{"total":96636764160,"available":96466522112,"internally_reserved":0,"allocated":101322752,"data_stored":99186411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150872,"kb_used_data":87744,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94220968,"statfs":{"total":96636764160,"available":96482271232,"internally_reserved":0,"allocated":89849856,"data_stored":87776120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68448256,"data_stored":68041796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46014464,"data_stored":45715931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31031296,"data_stored":30854903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86962176,"data_stored":86359083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98435072,"data_stored":97769374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81825792,"data_stored":81428482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44171264,"data_stored":43753450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70737920,"data_stored":70395951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:11:20.558 INFO:tasks.thrashosds.thrasher:Setting 3.19 to [3, 7, 1, 6] 2024-09-18T01:11:20.558 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.19', '3', '7', '1', '6'] 2024-09-18T01:11:20.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.19 3 7 1 6 2024-09-18T01:11:20.563 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:20.563 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:20.572 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:20.622 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:20.622+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:20.720 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.720 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:20.721 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:20.721 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:20.721 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:20.723 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:20.722+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:20.728 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:20.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:11:20.823 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:20.822+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:20.888 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.225459066, 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.225404841, 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22537238300001, 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22524112000001, 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22477113399999, 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22360853399999, 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22350125200001, 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.223419593, 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.22290420500001, 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 107.222084967, 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.64881988099999, 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.481960989000001, 2024-09-18T01:11:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.454115600999998, 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.454095301999999, 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.454075039000003, 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:20.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.454055186000005, 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:20.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:20.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:20.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 98.454036028999994, 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:20.978 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:20.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:20.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:20.984 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.070045602999997, 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:20.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:20.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 97.010086740000006, 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:20.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:20.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.225583294000003, 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:20.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:20.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:21.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.002 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:21.003 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:21.004 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:21.002+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:21.103 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:21.102+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:21.147 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:21.147 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:21.147 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:21.156 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:21.240 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:21.239+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:21.299 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:21.299 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:21.299 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:21.299 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:21.299 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:21.305 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:21.304+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:21.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:21.360 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:21.435 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:21.434+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:21.455 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793335284999998, 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:21.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793316089000001, 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:21.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793297727999999, 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:21.465 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:21.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793279623999993, 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:21.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793261027, 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:21.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793238027000001, 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:21.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:21.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793219355000005, 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:21.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793199352000002, 2024-09-18T01:11:21.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:21.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793181111999999, 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.493 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793162698000003, 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:21.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.793140671000003, 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788888892000003, 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:21.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.511 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788878440000005, 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788866748999993, 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.520 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788853451999998, 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:21.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:21.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788837380999993, 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.525 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:21.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.529 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788820723000001, 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:21.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788801427999999, 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.534 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:21.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:21.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788773198000001, 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.788752074000001, 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:21.547 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:21.549 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:21.549 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:21.549 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:21.550 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:21.550 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:21.550 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:21.550 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:21.551 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:21.547+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:21.560 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:21.561 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:21.577 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:21.578 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:21.594 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:21.594 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:21.594 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:21.611 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:21.611 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:21.614 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:21.614 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:21.649 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:21.648+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:21.667 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:21.668 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:21.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:21.751 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.19 pg_upmap_items mapping to [3->7,1->6] 2024-09-18T01:11:21.759 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:11:21.762 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:21.760+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:21.833 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:21.833 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:21.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:21.852 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:21.851+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:21.953 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:21.952+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:21.987 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:21.987 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:21.996 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:22.054 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:22.053+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:22.099 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.099 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.108 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:22.154 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:22.153+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:22.250 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.250 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.256 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:22.255+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:22.259 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:22.356 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:22.356+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:22.403 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.403 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.411 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:22.457 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:22.457+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:22.514 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.514 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:22.566 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:22.565+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:22.626 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.626 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:22.659 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:22.657+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:22.761 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:22.760+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:22.781 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.782 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:22.862 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:22.860+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:22.901 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:22.901 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:22.911 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:22.963 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:22.962+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:23.053 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.054 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:23.063 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:23.061+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:23.164 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:23.163+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:23.165 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.165 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.174 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:23.265 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:23.263+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:23.276 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.276 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.285 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:23.367 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:23.366+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:23.429 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.429 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:23.482 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:23.480+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:23.539 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.540 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.548 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:23.589 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:23.587+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:23.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:23.669 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:23.669+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:23.693 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:23.694 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:23.703 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:23.755 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:23.755 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:23.770 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:23.769+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:23.775 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:11:23.775 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:23.776 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:23.792 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:23.792 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:23.810 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:23.810 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:23.816 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:23.816 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:23.816 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:23.816 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:23.817 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:23.825 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:23.827 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:23.828 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:23.845 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:23.846 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:23.846 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:23.863 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:23.863 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:23.868 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:23.868 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:23.910 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:23.908+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:23.946 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.283571423, 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.283517198, 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.28348474000001, 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.28335347700001, 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.28288349100001, 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.28172089100001, 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:23.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.281613609, 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:23.976 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:23.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.28153195, 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.281016562, 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:23.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:23.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.280197324, 2024-09-18T01:11:23.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:23.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:23.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 104.70693223799999, 2024-09-18T01:11:23.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:23.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:24.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.540073346, 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.512227958, 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:24.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:24.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.512207659, 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.512187396, 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.022 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:24.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:24.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.512167543, 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:24.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 101.51214838600001, 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:24.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:24.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.12815796, 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:24.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:24.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.068199097, 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.047 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:24.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.283695651000002, 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:24.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.057 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:24.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:24.059 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:24.056+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:24.159 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:24.158+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:24.212 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:24.212 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:24.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:24.266 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:24.264+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:24.326 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:24.326 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:24.326 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:24.326 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:24.327 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:24.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:24.361 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:24.360+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:24.462 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:24.460+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.480 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817735145, 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.481 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:24.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817715949000004, 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:24.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817697588000001, 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:24.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:24.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.496 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817679483999996, 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.497 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:24.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.501 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817660887000002, 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:24.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817637887000004, 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817619214999993, 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:24.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817599212000005, 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:24.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:24.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817580972000002, 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:24.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817562558000006, 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:24.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.817540531000006, 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.531 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813288752000005, 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:24.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.540 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813278299999993, 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813266608999996, 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.546 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:24.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:24.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813253312000001, 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:24.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813237240999996, 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:24.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:24.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813220583000003, 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:24.559 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:24.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813201288000002, 2024-09-18T01:11:24.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:24.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:24.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:24.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:24.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813173058000004, 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:24.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:24.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.813151934000004, 2024-09-18T01:11:24.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:24.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:24.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:24.579 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:24.579 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:24.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:24.580 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:24.578+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:24.581 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:24.681 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:24.679+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:24.689 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:24.689 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:24.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:24.781 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:24.779+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:24.842 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:24.843 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:24.851 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:24.896 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:24.895+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:24.956 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:24.956 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:24.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:24.956 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 207.231638696, 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:24.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:24.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200612174, 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:24.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:24.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:24.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:24.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200602595999996, 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:24.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:24.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:24.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200588569999994, 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.979 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:24.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:24.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:24.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:24.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200559796999997, 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:24.987 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.988 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:24.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200532100999993, 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:24.994 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:24.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:24.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200497652999999, 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:25.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200467806999995, 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:25.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:25.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200445615999996, 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:25.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:25.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200408710000005, 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:25.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200387594999995, 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:25.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.200357069999995, 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:25.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:25.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:25.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194876816000004, 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:25.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194822957, 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:25.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:25.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194745534999996, 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.047 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:25.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:25.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194715420999998, 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:25.052 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:25.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:25.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194665502999996, 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.058 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:25.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:25.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:25.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194631916000006, 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:25.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:25.068 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194607382000001, 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:25.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:25.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:25.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.194422099999997, 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:25.074 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:25.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.080 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.080 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.081 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:25.082 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:25.080+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:25.182 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:25.180+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:25.224 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:25.224 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:25.283 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:25.281+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:25.336 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:25.336 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:25.390 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:25.389+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:25.449 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.848373425000005, 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.450 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:25.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:25.454 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847223409999998, 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:25.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:25.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847204098999995, 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.459 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:25.463 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847176352999995, 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:25.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847156501000001, 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:25.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847136637000006, 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:25.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847118430999998, 2024-09-18T01:11:25.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:25.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:25.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847098384000006, 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:25.482 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:25.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847080176000006, 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:25.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:25.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847062063999999, 2024-09-18T01:11:25.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:25.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847044146000002, 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.496 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.500 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.847026126000003, 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:25.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:25.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:25.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.505 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846473169999996, 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:25.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:25.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.510 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846454446999999, 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.511 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:25.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846424663999997, 2024-09-18T01:11:25.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:25.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:25.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846360575999995, 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:25.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:25.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846176275000005, 2024-09-18T01:11:25.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:25.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846158121000002, 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:25.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:25.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:25.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846129748999999, 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:25.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:25.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.846107822999997, 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:25.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.543 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.544 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:25.545 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:25.544+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:25.646 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:25.644+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:25.663 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.663 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:25.663 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:25.663 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:25.746 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:25.745+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:25.815 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.815 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:25.815 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:25.815 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:25.816 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:25.824 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:25.847 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:25.845+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:25.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 254.03106104, 2024-09-18T01:11:25.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:25.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:25.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:25.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:25.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:25.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:25.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 152.307110489, 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:25.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:25.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:25.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 116.339606022, 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.946 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:25.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.334633693000001, 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:25.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:25.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.329741310000003, 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:25.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.324326671999998, 2024-09-18T01:11:25.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:25.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:25.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:25.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.312752406000001, 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:25.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.303784712999999, 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.303730251000005, 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.298458877000002, 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:25.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.990 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.298378138000004, 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.297417139000004, 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:25.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:25.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:25.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.297358346999999, 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:26.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:26.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.296194378999999, 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:26.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:26.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.295829710000007, 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:26.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:26.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.017 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.295307174000001, 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:26.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:26.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.294909793000002, 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:26.025 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:26.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:26.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.293057063000006, 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 88.292934872000004, 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:26.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:26.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:258055 3.19 3:9ff6078a:::rbd_id.image_client.0-clone23:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e249)", 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.2894441940000001, 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95186631499999996, 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:26.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 258055 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639427+0000", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9410000000000001e-06 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639434+0000", 2024-09-18T01:11:26.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8730000000000001e-06 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639451+0000", 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6594000000000001e-05 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639458+0000", 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4789999999999999e-06 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639509+0000", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0645e-05 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639516+0000", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1810000000000004e-06 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588228+0000", 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94871193499999995 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588377+0000", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014855700000000001 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588500+0000", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000123497 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.590468+0000", 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019678320000000001 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591148+0000", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068034900000000001 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591217+0000", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8204e-05 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591292+0000", 2024-09-18T01:11:26.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5228000000000005e-05 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:26.048 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:26.049 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:26.049 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:26.050 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:26.050 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:26.051 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:26.049+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:26.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:26.051 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:26.052 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:26.052 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:26.052 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:26.061 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:26.061 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:26.079 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:26.079 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:26.080 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:26.096 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:26.096 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:26.099 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:26.100 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:26.150 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:26.148+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:26.155 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.155 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.164 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:26.250 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:26.248+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:26.306 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.306 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:26.351 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:26.350+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:26.452 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:26.452+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:26.457 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.457 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.466 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:26.553 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:26.551+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:26.568 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.568 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:26.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:11:26.662 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:26.660+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:26.731 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.731 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.741 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:26.755 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:26.754+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:26.769 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:26.769 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:26.769 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:11:26.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:11:26.856 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:26.854+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:26.887 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:26.887 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:26.897 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:26.917 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:11:26.960 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:26.958+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:27.058 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:27.056+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:27.063 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.063 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.073 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:27.094 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:11:27.160 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:27.158+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:27.239 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.239 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.249 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:27.260 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:27.259+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:27.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:11:27.353 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.354 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.361 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:27.359+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:27.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:27.449 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T01:11:27.461 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:27.461+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:27.510 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.510 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:27.562 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:27.561+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:27.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T01:11:27.595 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:11:27.632 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.632 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:27.685 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:27.683+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:27.709 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-18T01:11:27.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T01:11:27.794 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:27.792+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:27.795 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.795 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:27.864 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:27.862+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:27.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:11:27.910 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:27.910 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:27.918 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:27.966 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:27.965+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:28.020 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:28.021 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:28.029 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:28.066 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:28.065+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:28.101 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:28.167 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:28.165+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:28.171 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:28.171 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:28.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:28.220 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:28.220 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:28.236 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:28.236 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:28.252 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:28.252 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:28.268 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:28.268 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:28.266+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:28.283 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:28.283 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:28.284 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:28.285 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:28.292 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:28.301 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:28.301 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:28.301 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:28.318 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:28.318 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:28.321 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:28.321 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:28.369 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:28.368+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:28.395 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:28.395 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:28.403 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:28.470 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:28.469+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:28.549 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:28.549 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:28.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.947757801999998, 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:28.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:28.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946607787000005, 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:28.554 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:28.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946588476000002, 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.559 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:28.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:28.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946560730000002, 2024-09-18T01:11:28.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:28.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.567 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946540877999993, 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.568 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:28.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:28.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:28.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946521013999998, 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:28.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946502808000005, 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:28.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:28.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:28.581 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946482760999999, 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.585 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946464552999998, 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.586 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:28.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:28.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:28.590 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946446441000006, 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:28.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:28.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946428522999994, 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:28.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.599 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.946410502999996, 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:28.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:28.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945857547000003, 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:28.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:28.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945838824000006, 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:28.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:28.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945809041000004, 2024-09-18T01:11:28.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945744953000002, 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.618 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:28.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945560651999998, 2024-09-18T01:11:28.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945542497999995, 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:28.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:28.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945514126000006, 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.945492200000004, 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:28.640 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:28.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:28.642 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:28.640+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:28.742 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:28.741+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:28.790 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:28.790 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:28.790 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:28.790 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:28.799 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:28.844 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:28.842+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:28.941 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:28.941 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:28.942 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:28.942 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:28.942 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:28.944 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:28.942+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:28.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:29.045 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:29.044+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 257.156244249, 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 155.432293698, 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.464789231, 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.459816902, 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.454924519000002, 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.449509880999997, 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:29.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:29.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.437935615000001, 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.428967921999998, 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.095 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:29.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.428913460000004, 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:29.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:29.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:29.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.423642086000001, 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.423561347000003, 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.422600348000003, 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.422541555999999, 2024-09-18T01:11:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.421377587999999, 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:29.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.421012919000006, 2024-09-18T01:11:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.420490383000001, 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.420093002000002, 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.418240272000006, 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.418118081000003, 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:258055 3.19 3:9ff6078a:::rbd_id.image_client.0-clone23:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e249)", 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4146274029999999, 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95186631499999996, 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 258055 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639427+0000", 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9410000000000001e-06 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639434+0000", 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8730000000000001e-06 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639451+0000", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6594000000000001e-05 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639458+0000", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4789999999999999e-06 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639509+0000", 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0645e-05 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639516+0000", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1810000000000004e-06 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588228+0000", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94871193499999995 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588377+0000", 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014855700000000001 2024-09-18T01:11:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588500+0000", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000123497 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.590468+0000", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019678320000000001 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591148+0000", 2024-09-18T01:11:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068034900000000001 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591217+0000", 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8204e-05 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591292+0000", 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5228000000000005e-05 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:29.165 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:29.166 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:29.166 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:29.164+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:29.267 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:29.266+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:29.270 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:29.270 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:29.270 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:29.270 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:29.279 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:29.368 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:29.366+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:29.381 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:29.381 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:29.381 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:29.381 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:29.381 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:29.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:29.470 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:29.469+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 211.810181129, 2024-09-18T01:11:29.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:29.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:29.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.786723038999995, 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:29.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:29.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:29.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.773644786999995, 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.548 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:29.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.773624530000006, 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:29.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.773279494999997, 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:29.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772972365000001, 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:29.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772936606000002, 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:29.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:29.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:29.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772898337000001, 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:29.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:29.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:29.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772873864999994, 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:29.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:29.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:29.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772853204, 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.585 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:29.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772834579999994, 2024-09-18T01:11:29.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772814788999995, 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:29.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772795685999995, 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:29.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772777595999997, 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:29.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772759496999996, 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:29.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:29.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772741167999996, 2024-09-18T01:11:29.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:29.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:29.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772722802000004, 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772704542, 2024-09-18T01:11:29.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:29.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772686088, 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:29.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:29.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:29.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.772636402000003, 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:29.640 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:29.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:29.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:29.646 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:29.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:11:29.647 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:29.646+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:29.648 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:29.747 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:29.745+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:29.799 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:29.800 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:29.800 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:29.810 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:29.851 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:29.849+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:29.949 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:29.946+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:29.954 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:29.954 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:29.963 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:30.050 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:30.049+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:30.107 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.194106841000007, 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.108 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:30.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.193898809999993, 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:30.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179862235000002, 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:30.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:30.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:30.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.121 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179838887000003, 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179817365999995, 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.126 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:30.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:30.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179796647000003, 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:30.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:30.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:30.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179775871000004, 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:30.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.139 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179755725999996, 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179735350000001, 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.144 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:30.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179714622000006, 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.148 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:30.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:30.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179690163999993, 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:30.152 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:30.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.179669687000001, 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.157 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170893491000001, 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:30.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:30.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170868490999993, 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.166 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.169 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170831336000006, 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:30.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:30.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:30.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170804653000005, 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:30.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170784753999996, 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.181 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170765547000002, 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170746534000003, 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.189 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.170728260999994, 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:30.193 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:30.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:30.195 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:30.193+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:30.296 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:30.295+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:30.298 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:30.298 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:30.299 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:30.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:30.322 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:30.396 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:30.394+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:30.412 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:30.412 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:30.412 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:30.412 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:30.412 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:30.421 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:30.472 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:30.472 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:30.489 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:30.496 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:30.495+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:30.505 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:30.506 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:30.522 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:30.522 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:30.539 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:30.539 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:30.556 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:30.567 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:30.567 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:30.574 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:30.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:30.577 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:30.578 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:30.597 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:30.597+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:30.621 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:11:30.699 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:30.697+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:30.736 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:30.736 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:30.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:30.799 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:30.798+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:30.889 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:30.889 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:30.898 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:30.899 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:30.897+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:31.000 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:30.998+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:31.000 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.000 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.009 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:31.101 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:31.099+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:31.115 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.115 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.124 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:31.202 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:31.201+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:31.228 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.229 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.237 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:31.302 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:31.301+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:31.379 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.379 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.388 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:31.403 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:31.402+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:31.491 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.492 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:31.505 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:31.503+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:31.603 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.603 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.605 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:31.604+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:31.612 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:31.707 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:31.705+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:31.762 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.762 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.771 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:31.808 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:31.806+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:31.873 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.874 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.882 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:31.908 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:31.906+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:31.985 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:31.985 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:31.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:32.009 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:32.008+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:32.096 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:32.096 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:32.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:32.149 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:32.147+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:32.211 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:32.211 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:32.211 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:32.211+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:32.219 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:32.312 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:32.312+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:32.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:32.415 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:32.413+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:32.474 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:32.474 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:32.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:32.515 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:32.514+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:32.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:32.623 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:32.621+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:32.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:11:32.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:32.716 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:32.717 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:32.717 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:32.715+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:32.736 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:32.758 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:32.758 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:32.776 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:32.776 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:32.793 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:32.793 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:32.803 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:32.803 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:32.803 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:32.803 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:32.803 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:32.809 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:32.810 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:32.810 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:32.812 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:32.827 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:32.827 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:32.827 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:32.825+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:32.830 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:32.830 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:32.918 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:32.916+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.018399138999996, 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.933 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:32.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.937 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.018191107999996, 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.938 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.942 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004154533000005, 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:32.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:32.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004131185000006, 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:32.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:32.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004109663999998, 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004088945000007, 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:32.956 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:32.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004068169000007, 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.961 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:32.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004048023999999, 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:32.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004027648000005, 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:32.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:32.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.004006919999995, 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.975 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:32.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:32.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:32.979 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.003982461999996, 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:32.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.983 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.003961985000004, 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.984 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:32.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:32.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:32.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995185789000004, 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:32.995 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995160788999996, 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:32.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:32.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:32.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995123633999995, 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:33.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:33.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995096950999994, 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:33.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995077051999999, 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:33.009 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:33.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.013 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995057845000005, 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:33.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995038832000006, 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.018 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:33.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.995020558999997, 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.022 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.026 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:33.028 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:33.026+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:33.062 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:33.063 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:33.063 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:11:33.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:11:33.128 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:33.126+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:33.269 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:33.267+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:33.327 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.328 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:33.328 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:33.328 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:33.328 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.330 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:33.328+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:33.336 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:33.356 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:11:33.356 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.356 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 257, 2024-09-18T01:11:33.356 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "e017e0c5-d4e1-4f77-8127-47c02525e65d", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-18T01:06:59.746320+0000", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-18T01:11:32.620999+0000", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-18T01:10:54.570849+0000", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-18T01:10:43.698009+0000", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5806080, 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-18T01:11:33.357 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 3, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-18T01:11:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:10.752532+0000", 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:33.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:33.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-18T01:11:33.361 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:33.362 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:33.363 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:33.364 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "rbd", 2024-09-18T01:11:33.365 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:17.736243+0000", 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:33.366 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 8, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 8, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 8, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 8, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 8, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.367 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 2, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 21, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:33.368 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:33.369 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.370 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 3, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 3, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.75, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2.25, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2.25, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "pool_client.0", 2024-09-18T01:11:33.371 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-18T01:07:27.050688+0000", 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 8193, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:11:33.372 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-18T01:11:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "257", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 99, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 257, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-18T01:11:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "options": {}, 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd": {} 2024-09-18T01:11:33.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.9900000095367432, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.9900000095367432, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.75, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.75, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.377 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "13ff2fc9-b863-45cb-b111-62d2ff06dff1", 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.378 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1379550312 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1379550312", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1379550312", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1379550312", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1379550312", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "01867b0a-aeb8-4b92-897b-f8176abf95a9", 2024-09-18T01:11:33.383 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 248, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2892053774 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2892053774", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2892053774", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2892053774", 2024-09-18T01:11:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2892053774", 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "788e3030-dfc0-45b5-9c99-763dc9283283", 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-18T01:11:33.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-18T01:11:33.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-18T01:11:33.392 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2679997431 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/2679997431", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/2679997431", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/2679997431", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/2679997431", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "9ecd2117-0d0b-4629-9e54-3578a68fa0db", 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.395 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.396 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 712549013 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/712549013", 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/712549013", 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/712549013", 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/712549013", 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.400 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "3af2d4ee-bce6-49df-a10e-f7c367d9fcb5", 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 219, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6816", 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6818", 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6820", 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6822", 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.403 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6828", 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6830", 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6824", 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6826", 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3636718449 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6818/3636718449", 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6822/3636718449", 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6830/3636718449", 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6826/3636718449", 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:33.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "fbac8f09-7655-4b5e-8bb5-74650aa3c2f1", 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 208, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6817", 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6819", 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.408 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6821", 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6823", 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.409 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6829", 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6831", 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.410 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6825", 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6827", 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3011845137 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.411 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6819/3011845137", 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6823/3011845137", 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6831/3011845137", 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6827/3011845137", 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1a98a211-cab7-414a-ad17-b06cede06bcf", 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 0, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 205, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 219, 2024-09-18T01:11:33.413 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6808", 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6809", 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6810", 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6811", 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6814", 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6815", 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6812", 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6813", 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 4172964327 2024-09-18T01:11:33.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6809/4172964327", 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6811/4172964327", 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6815/4172964327", 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6813/4172964327", 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "exists" 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:33.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "494f3743-c1f1-410f-b5f8-ce5927494495", 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 15, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 249, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6800", 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6801", 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-18T01:11:33.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6802", 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6803", 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-18T01:11:33.421 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6806", 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6807", 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-18T01:11:33.422 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6804", 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.162:6805", 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3795653175 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.423 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.162:6801/3795653175", 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.162:6803/3795653175", 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.162:6807/3795653175", 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.162:6805/3795653175", 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-18T01:11:33.424 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.476040+0000", 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.425 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.253614+0000", 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.779954+0000", 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.553728+0000", 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.427 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.609031+0000", 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.157009+0000", 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:09.118047+0000", 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 219 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-18T01:07:08.926014+0000", 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-18T01:11:33.430 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.3", 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 4, 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.19", 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.431 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 7 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [ 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.432 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.19", 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: 7, 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: 1 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-18T01:11:33.433 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [ 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 36, 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 38, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 40, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 42, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.435 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 44, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 46, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 48, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 50, 2024-09-18T01:11:33.436 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 52, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 54, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 56, 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 58, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 60, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 62, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 64, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 66, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 68, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.439 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 70, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 72, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 74, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 76, 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 78, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 80, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 82, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 84, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 86, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 88, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.442 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 90, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 92, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 94, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 96, 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 98, 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 2 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [ 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: "snaps": [ 2024-09-18T01:11:33.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "begin": 36, 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "length": 1 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-18T01:11:33.445 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:11:33.446 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:11:33.446 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-18T01:11:33.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.446 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.448 INFO:tasks.thrashosds.thrasher:j is {'epoch': 257, 'fsid': 'e017e0c5-d4e1-4f77-8127-47c02525e65d', 'created': '2024-09-18T01:06:59.746320+0000', 'modified': '2024-09-18T01:11:32.620999+0000', 'last_up_change': '2024-09-18T01:10:54.570849+0000', 'last_in_change': '2024-09-18T01:10:43.698009+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:07:10.752532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:07:17.736243+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:07:27.050688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 99, 'snap_epoch': 257, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '13ff2fc9-b863-45cb-b111-62d2ff06dff1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1379550312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1379550312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1379550312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1379550312}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1379550312}]}, 'public_addr': '172.21.15.120:6825/1379550312', 'cluster_addr': '172.21.15.120:6827/1379550312', 'heartbeat_back_addr': '172.21.15.120:6831/1379550312', 'heartbeat_front_addr': '172.21.15.120:6829/1379550312', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '01867b0a-aeb8-4b92-897b-f8176abf95a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2892053774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2892053774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2892053774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2892053774}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2892053774}]}, 'public_addr': '172.21.15.120:6817/2892053774', 'cluster_addr': '172.21.15.120:6819/2892053774', 'heartbeat_back_addr': '172.21.15.120:6823/2892053774', 'heartbeat_front_addr': '172.21.15.120:6821/2892053774', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '788e3030-dfc0-45b5-9c99-763dc9283283', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 2679997431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 2679997431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 2679997431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 2679997431}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 2679997431}]}, 'public_addr': '172.21.15.120:6809/2679997431', 'cluster_addr': '172.21.15.120:6811/2679997431', 'heartbeat_back_addr': '172.21.15.120:6815/2679997431', 'heartbeat_front_addr': '172.21.15.120:6813/2679997431', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ecd2117-0d0b-4629-9e54-3578a68fa0db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 712549013}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 712549013}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 712549013}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 712549013}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 712549013}]}, 'public_addr': '172.21.15.120:6801/712549013', 'cluster_addr': '172.21.15.120:6803/712549013', 'heartbeat_back_addr': '172.21.15.120:6807/712549013', 'heartbeat_front_addr': '172.21.15.120:6805/712549013', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3af2d4ee-bce6-49df-a10e-f7c367d9fcb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 219, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6818', 'nonce': 3636718449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6822', 'nonce': 3636718449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6830', 'nonce': 3636718449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 3636718449}, {'type': 'v1', 'addr': '172.21.15.162:6826', 'nonce': 3636718449}]}, 'public_addr': '172.21.15.162:6818/3636718449', 'cluster_addr': '172.21.15.162:6822/3636718449', 'heartbeat_back_addr': '172.21.15.162:6830/3636718449', 'heartbeat_front_addr': '172.21.15.162:6826/3636718449', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'fbac8f09-7655-4b5e-8bb5-74650aa3c2f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6817', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 3011845137}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6821', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 3011845137}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6829', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 3011845137}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6825', 'nonce': 3011845137}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 3011845137}]}, 'public_addr': '172.21.15.162:6819/3011845137', 'cluster_addr': '172.21.15.162:6823/3011845137', 'heartbeat_back_addr': '172.21.15.162:6831/3011845137', 'heartbeat_front_addr': '172.21.15.162:6827/3011845137', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1a98a211-cab7-414a-ad17-b06cede06bcf', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 205, 'down_at': 219, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 4172964327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 4172964327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 4172964327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 4172964327}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 4172964327}]}, 'public_addr': '172.21.15.162:6809/4172964327', 'cluster_addr': '172.21.15.162:6811/4172964327', 'heartbeat_back_addr': '172.21.15.162:6815/4172964327', 'heartbeat_front_addr': '172.21.15.162:6813/4172964327', 'state': ['exists']}, {'osd': 7, 'uuid': '494f3743-c1f1-410f-b5f8-ce5927494495', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 249, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 3795653175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 3795653175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 3795653175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 3795653175}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 3795653175}]}, 'public_addr': '172.21.15.162:6801/3795653175', 'cluster_addr': '172.21.15.162:6803/3795653175', 'heartbeat_back_addr': '172.21.15.162:6807/3795653175', 'heartbeat_front_addr': '172.21.15.162:6805/3795653175', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.476040+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.253614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.779954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.553728+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.609031+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.157009+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:09.118047+0000', 'dead_epoch': 219}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:07:08.926014+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 6}]}, {'pgid': '3.19', 'mappings': [{'from': 3, 'to': 7}, {'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.19', 'osds': [7, 1]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}, {'begin': 38, 'length': 1}, {'begin': 40, 'length': 1}, {'begin': 42, 'length': 1}, {'begin': 44, 'length': 1}, {'begin': 46, 'length': 1}, {'begin': 48, 'length': 1}, {'begin': 50, 'length': 1}, {'begin': 52, 'length': 1}, {'begin': 54, 'length': 1}, {'begin': 56, 'length': 1}, {'begin': 58, 'length': 1}, {'begin': 60, 'length': 1}, {'begin': 62, 'length': 1}, {'begin': 64, 'length': 1}, {'begin': 66, 'length': 1}, {'begin': 68, 'length': 1}, {'begin': 70, 'length': 1}, {'begin': 72, 'length': 1}, {'begin': 74, 'length': 1}, {'begin': 76, 'length': 1}, {'begin': 78, 'length': 1}, {'begin': 80, 'length': 1}, {'begin': 82, 'length': 1}, {'begin': 84, 'length': 1}, {'begin': 86, 'length': 1}, {'begin': 88, 'length': 1}, {'begin': 90, 'length': 1}, {'begin': 92, 'length': 1}, {'begin': 94, 'length': 1}, {'begin': 96, 'length': 1}, {'begin': 98, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:11:33.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:11:33.449 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:33.446+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:33.548 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:33.546+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:33.552 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.889864645, 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:33.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:33.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.88981042, 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.558 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:33.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.889777962, 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:33.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.889646699, 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:33.568 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:33.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:33.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.889176713, 2024-09-18T01:11:33.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:33.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.888014113, 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:33.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:33.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.887906831, 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:33.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:33.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.88782517200001, 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:33.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:33.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:33.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.887309784, 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:33.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:33.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.886490546, 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:33.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.31322546, 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:33.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:33.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.146366568, 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:33.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:33.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.11852118, 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:33.618 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:33.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:33.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:33.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:33.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.118500881, 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:33.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:33.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:33.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:33.630 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.11848061800001, 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:33.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:33.636 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.11846076499999, 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:33.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:33.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 111.118441608, 2024-09-18T01:11:33.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:33.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:33.648 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.734451182, 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:33.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.674492319, 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:33.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:33.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.889988872999993, 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:33.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:33.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:33.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:33.665 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:33.667 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:11:33.668 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:33.664+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:33.767 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:33.765+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:33.796 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:11:33.797 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:11:33.809 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-18T01:11:33.631279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636107928,"num_objects":687,"num_object_clones":0,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":687,"num_whiteouts":0,"num_read":262492,"num_read_kb":1259683,"num_write":35100,"num_write_kb":852180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2418,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24666,"ondisk_log_size":24666,"up":75,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":994784,"kb_used_data":503028,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":753979936,"statfs":{"total":773094113280,"available":772075454464,"internally_reserved":0,"allocated":515100672,"data_stored":500758298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502648368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-165595929,"num_objects":-64,"num_object_clones":0,"num_object_copies":-128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":-6,"num_objects_unfound":0,"num_objects_dirty":-64,"num_whiteouts":0,"num_read":1006,"num_read_kb":860,"num_write":218,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022754"},"pg_stats":[{"pgid":"3.1f","version":"255'512","reported_seq":7148,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.630380+0000","last_change":"2024-09-18T01:11:32.630622+0000","last_active":"2024-09-18T01:11:32.630380+0000","last_peered":"2024-09-18T01:11:32.630380+0000","last_clean":"2024-09-18T01:11:32.630380+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:11:32.630380+0000","last_undegraded":"2024-09-18T01:11:32.630380+0000","last_fullsized":"2024-09-18T01:11:32.630380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:46.094845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022687390000000001,"stat_sum":{"num_bytes":20862296,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":6275,"num_read_kb":39533,"num_write":611,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1e","version":"255'724","reported_seq":4283,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:30.743315+0000","last_change":"2024-09-18T01:10:55.580235+0000","last_active":"2024-09-18T01:11:30.743315+0000","last_peered":"2024-09-18T01:11:30.743315+0000","last_clean":"2024-09-18T01:10:53.747252+0000","last_became_active":"2024-09-18T01:10:55.580235+0000","last_became_peered":"2024-09-18T01:10:55.580235+0000","last_unstale":"2024-09-18T01:11:30.743315+0000","last_undegraded":"2024-09-18T01:10:55.578208+0000","last_fullsized":"2024-09-18T01:10:55.578105+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:29.023574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058770700000000003,"stat_sum":{"num_bytes":10656621,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4849,"num_read_kb":33903,"num_write":1113,"num_write_kb":25803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":19}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"256'640","reported_seq":4626,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643790+0000","last_change":"2024-09-18T01:11:32.643993+0000","last_active":"2024-09-18T01:11:32.643790+0000","last_peered":"2024-09-18T01:11:32.643790+0000","last_clean":"2024-09-18T01:11:32.643790+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:11:32.643790+0000","last_undegraded":"2024-09-18T01:11:32.643790+0000","last_fullsized":"2024-09-18T01:11:32.643790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:34.764786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070234200000000001,"stat_sum":{"num_bytes":24317679,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":959,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1c","version":"257'658","reported_seq":5875,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.688925+0000","last_change":"2024-09-18T01:11:32.643738+0000","last_active":"2024-09-18T01:11:32.688925+0000","last_peered":"2024-09-18T01:11:32.688925+0000","last_clean":"2024-09-18T01:11:32.688925+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:11:32.688925+0000","last_undegraded":"2024-09-18T01:11:32.688925+0000","last_fullsized":"2024-09-18T01:11:32.688925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:19.990129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482168,"stat_sum":{"num_bytes":12615453,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":5436,"num_read_kb":39217,"num_write":876,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1b","version":"257'1246","reported_seq":7500,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.752089+0000","last_change":"2024-09-18T01:11:32.630177+0000","last_active":"2024-09-18T01:11:32.752089+0000","last_peered":"2024-09-18T01:11:32.752089+0000","last_clean":"2024-09-18T01:11:32.752089+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:11:32.752089+0000","last_undegraded":"2024-09-18T01:11:32.752089+0000","last_fullsized":"2024-09-18T01:11:32.752089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1246,"log_dups_size":0,"ondisk_log_size":1246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:39.722881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019602399999999998,"stat_sum":{"num_bytes":24953868,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":10068,"num_read_kb":42514,"num_write":1772,"num_write_kb":30434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.1a","version":"255'476","reported_seq":7414,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.730144+0000","last_change":"2024-09-18T01:11:30.632884+0000","last_active":"2024-09-18T01:11:30.730144+0000","last_peered":"2024-09-18T01:11:30.730144+0000","last_clean":"2024-09-18T01:11:30.730144+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:11:30.730144+0000","last_undegraded":"2024-09-18T01:11:30.730144+0000","last_fullsized":"2024-09-18T01:11:30.730144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:19:19.587520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059022800000000004,"stat_sum":{"num_bytes":15596178,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":6299,"num_read_kb":37993,"num_write":607,"num_write_kb":25987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.19","version":"257'681","reported_seq":9452,"reported_epoch":257,"state":"active+clean+remapped","last_fresh":"2024-09-18T01:11:32.687482+0000","last_change":"2024-09-18T01:11:32.635155+0000","last_active":"2024-09-18T01:11:32.687482+0000","last_peered":"2024-09-18T01:11:32.687482+0000","last_clean":"2024-09-18T01:11:32.687482+0000","last_became_active":"2024-09-18T01:11:25.585751+0000","last_became_peered":"2024-09-18T01:11:25.585751+0000","last_unstale":"2024-09-18T01:11:32.687482+0000","last_undegraded":"2024-09-18T01:11:32.687482+0000","last_fullsized":"2024-09-18T01:11:32.687482+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:53.793262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872891,"stat_sum":{"num_bytes":20372064,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":768,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":20}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.18","version":"255'567","reported_seq":13327,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.621412+0000","last_change":"2024-09-18T01:11:30.621412+0000","last_active":"2024-09-18T01:11:30.621412+0000","last_peered":"2024-09-18T01:11:30.621412+0000","last_clean":"2024-09-18T01:11:30.621412+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:11:30.621412+0000","last_undegraded":"2024-09-18T01:11:30.621412+0000","last_fullsized":"2024-09-18T01:11:30.621412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:52.905259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086091100000000001,"stat_sum":{"num_bytes":29763459,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":12952,"num_read_kb":57829,"num_write":662,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.17","version":"255'1196","reported_seq":7636,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.630464+0000","last_change":"2024-09-18T01:11:32.630656+0000","last_active":"2024-09-18T01:11:32.630464+0000","last_peered":"2024-09-18T01:11:32.630464+0000","last_clean":"2024-09-18T01:11:32.630464+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:11:32.630464+0000","last_undegraded":"2024-09-18T01:11:32.630464+0000","last_fullsized":"2024-09-18T01:11:32.630464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1196,"log_dups_size":0,"ondisk_log_size":1196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:50:04.343824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022029649999999999,"stat_sum":{"num_bytes":29119086,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":9121,"num_read_kb":41654,"num_write":1920,"num_write_kb":28915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.16","version":"255'697","reported_seq":4581,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643366+0000","last_change":"2024-09-18T01:11:32.643590+0000","last_active":"2024-09-18T01:11:32.643366+0000","last_peered":"2024-09-18T01:11:32.643366+0000","last_clean":"2024-09-18T01:11:32.643366+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:11:32.643366+0000","last_undegraded":"2024-09-18T01:11:32.643366+0000","last_fullsized":"2024-09-18T01:11:32.643366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:07:53.076908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000417427,"stat_sum":{"num_bytes":25668979,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":957,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.15","version":"256'915","reported_seq":9809,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.633658+0000","last_change":"2024-09-18T01:11:32.633870+0000","last_active":"2024-09-18T01:11:32.633658+0000","last_peered":"2024-09-18T01:11:32.633658+0000","last_clean":"2024-09-18T01:11:32.633658+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:11:32.633658+0000","last_undegraded":"2024-09-18T01:11:32.633658+0000","last_fullsized":"2024-09-18T01:11:32.633658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":915,"log_dups_size":0,"ondisk_log_size":915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:57.475643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000710694,"stat_sum":{"num_bytes":28250776,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":10723,"num_read_kb":60463,"num_write":1282,"num_write_kb":30043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.14","version":"255'658","reported_seq":6379,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.633057+0000","last_change":"2024-09-18T01:11:30.633057+0000","last_active":"2024-09-18T01:11:30.633057+0000","last_peered":"2024-09-18T01:11:30.633057+0000","last_clean":"2024-09-18T01:11:30.633057+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:11:30.633057+0000","last_undegraded":"2024-09-18T01:11:30.633057+0000","last_fullsized":"2024-09-18T01:11:30.633057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:01.217283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067308799999999996,"stat_sum":{"num_bytes":11083907,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":6536,"num_read_kb":23575,"num_write":950,"num_write_kb":16106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":257,"state":"active+undersized","last_fresh":"2024-09-18T01:11:32.633064+0000","last_change":"2024-09-18T01:10:55.580802+0000","last_active":"2024-09-18T01:11:32.633064+0000","last_peered":"2024-09-18T01:11:32.633064+0000","last_clean":"2024-09-18T01:09:55.569213+0000","last_became_active":"2024-09-18T01:10:55.580802+0000","last_became_peered":"2024-09-18T01:10:55.580802+0000","last_unstale":"2024-09-18T01:11:32.633064+0000","last_undegraded":"2024-09-18T01:11:32.633064+0000","last_fullsized":"2024-09-18T01:10:55.579666+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:34.355623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"257'720","reported_seq":4604,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.630258+0000","last_change":"2024-09-18T01:11:32.630258+0000","last_active":"2024-09-18T01:11:32.630258+0000","last_peered":"2024-09-18T01:11:32.630258+0000","last_clean":"2024-09-18T01:11:32.630258+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:11:32.630258+0000","last_undegraded":"2024-09-18T01:11:32.630258+0000","last_fullsized":"2024-09-18T01:11:32.630258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:38:07.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020174659999999999,"stat_sum":{"num_bytes":10101919,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4852,"num_read_kb":27344,"num_write":974,"num_write_kb":22612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":494,"reported_epoch":257,"state":"active+undersized","last_fresh":"2024-09-18T01:11:32.624572+0000","last_change":"2024-09-18T01:10:55.576107+0000","last_active":"2024-09-18T01:11:32.624572+0000","last_peered":"2024-09-18T01:11:32.624572+0000","last_clean":"2024-09-18T01:10:53.742769+0000","last_became_active":"2024-09-18T01:10:55.576107+0000","last_became_peered":"2024-09-18T01:10:55.576107+0000","last_unstale":"2024-09-18T01:11:32.624572+0000","last_undegraded":"2024-09-18T01:11:32.624572+0000","last_fullsized":"2024-09-18T01:10:55.575499+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:34.049490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"255'471","reported_seq":7635,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.700584+0000","last_change":"2024-09-18T01:11:30.621363+0000","last_active":"2024-09-18T01:11:30.700584+0000","last_peered":"2024-09-18T01:11:30.700584+0000","last_clean":"2024-09-18T01:11:30.700584+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:11:30.700584+0000","last_undegraded":"2024-09-18T01:11:30.700584+0000","last_fullsized":"2024-09-18T01:11:30.700584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:27.390586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065836599999999996,"stat_sum":{"num_bytes":14968993,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":725,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":492,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.633375+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:11:32.633375+0000","last_peered":"2024-09-18T01:11:32.633375+0000","last_clean":"2024-09-18T01:11:32.633375+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:11:32.633375+0000","last_undegraded":"2024-09-18T01:11:32.633375+0000","last_fullsized":"2024-09-18T01:11:32.633375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"257'542","reported_seq":6999,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.625382+0000","last_change":"2024-09-18T01:11:32.625382+0000","last_active":"2024-09-18T01:11:32.625382+0000","last_peered":"2024-09-18T01:11:32.625382+0000","last_clean":"2024-09-18T01:11:32.625382+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:11:32.625382+0000","last_undegraded":"2024-09-18T01:11:32.625382+0000","last_fullsized":"2024-09-18T01:11:32.625382+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:37.021973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049260899999999997,"stat_sum":{"num_bytes":21804874,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5659,"num_read_kb":52458,"num_write":699,"num_write_kb":28694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":493,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.625130+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:11:32.625130+0000","last_peered":"2024-09-18T01:11:32.625130+0000","last_clean":"2024-09-18T01:11:32.625130+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:11:32.625130+0000","last_undegraded":"2024-09-18T01:11:32.625130+0000","last_fullsized":"2024-09-18T01:11:32.625130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"253'926","reported_seq":12694,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643646+0000","last_change":"2024-09-18T01:11:32.643837+0000","last_active":"2024-09-18T01:11:32.643646+0000","last_peered":"2024-09-18T01:11:32.643646+0000","last_clean":"2024-09-18T01:11:32.643646+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:11:32.643646+0000","last_undegraded":"2024-09-18T01:11:32.643646+0000","last_fullsized":"2024-09-18T01:11:32.643646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:07.566928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054719299999999996,"stat_sum":{"num_bytes":41103835,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":13974,"num_read_kb":64842,"num_write":1416,"num_write_kb":38731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.3","version":"257'620","reported_seq":8823,"reported_epoch":257,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:32.730065+0000","last_change":"2024-09-18T01:10:55.580459+0000","last_active":"2024-09-18T01:11:32.730065+0000","last_peered":"2024-09-18T01:11:32.730065+0000","last_clean":"2024-09-18T01:10:50.027978+0000","last_became_active":"2024-09-18T01:10:55.580459+0000","last_became_peered":"2024-09-18T01:10:55.580459+0000","last_unstale":"2024-09-18T01:11:32.730065+0000","last_undegraded":"2024-09-18T01:10:55.578156+0000","last_fullsized":"2024-09-18T01:10:55.578070+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":33,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:59.701288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039494299999999998,"stat_sum":{"num_bytes":9438348,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":8256,"num_read_kb":43588,"num_write":824,"num_write_kb":24115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":494,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643126+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:11:32.643126+0000","last_peered":"2024-09-18T01:11:32.643126+0000","last_clean":"2024-09-18T01:11:32.643126+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:11:32.643126+0000","last_undegraded":"2024-09-18T01:11:32.643126+0000","last_fullsized":"2024-09-18T01:11:32.643126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"256'680","reported_seq":13448,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.625589+0000","last_change":"2024-09-18T01:11:32.625589+0000","last_active":"2024-09-18T01:11:32.625589+0000","last_peered":"2024-09-18T01:11:32.625589+0000","last_clean":"2024-09-18T01:11:32.625589+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:11:32.625589+0000","last_undegraded":"2024-09-18T01:11:32.625589+0000","last_fullsized":"2024-09-18T01:11:32.625589+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:37.057145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056937100000000005,"stat_sum":{"num_bytes":28013935,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":12625,"num_read_kb":60625,"num_write":1038,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.624722+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:11:32.624722+0000","last_peered":"2024-09-18T01:11:32.624722+0000","last_clean":"2024-09-18T01:11:32.624722+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:11:32.624722+0000","last_undegraded":"2024-09-18T01:11:32.624722+0000","last_fullsized":"2024-09-18T01:11:32.624722+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"257'864","reported_seq":16000,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.628545+0000","last_change":"2024-09-18T01:11:32.628545+0000","last_active":"2024-09-18T01:11:32.628545+0000","last_peered":"2024-09-18T01:11:32.628545+0000","last_clean":"2024-09-18T01:11:32.628545+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:11:32.628545+0000","last_undegraded":"2024-09-18T01:11:32.628545+0000","last_fullsized":"2024-09-18T01:11:32.628545+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:39:58.318284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046200499999999998,"stat_sum":{"num_bytes":28202890,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":16756,"num_read_kb":44724,"num_write":1005,"num_write_kb":34546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":492,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.632912+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:11:32.632912+0000","last_peered":"2024-09-18T01:11:32.632912+0000","last_clean":"2024-09-18T01:11:32.632912+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:11:32.632912+0000","last_undegraded":"2024-09-18T01:11:32.632912+0000","last_fullsized":"2024-09-18T01:11:32.632912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"250'484","reported_seq":8289,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.620563+0000","last_change":"2024-09-18T01:11:30.620842+0000","last_active":"2024-09-18T01:11:30.620563+0000","last_peered":"2024-09-18T01:11:30.620563+0000","last_clean":"2024-09-18T01:11:30.620563+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:11:30.620563+0000","last_undegraded":"2024-09-18T01:11:30.620563+0000","last_fullsized":"2024-09-18T01:11:30.620563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:00:51.961731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052122500000000005,"stat_sum":{"num_bytes":19203626,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7192,"num_read_kb":33150,"num_write":571,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"2.3","version":"19'1","reported_seq":529,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643047+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:11:32.643047+0000","last_peered":"2024-09-18T01:11:32.643047+0000","last_clean":"2024-09-18T01:11:32.643047+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:11:32.643047+0000","last_undegraded":"2024-09-18T01:11:32.643047+0000","last_fullsized":"2024-09-18T01:11:32.643047+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":644,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.632933+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:11:32.632933+0000","last_peered":"2024-09-18T01:11:32.632933+0000","last_clean":"2024-09-18T01:11:32.632933+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:11:32.632933+0000","last_undegraded":"2024-09-18T01:11:32.632933+0000","last_fullsized":"2024-09-18T01:11:32.632933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"255'1245","reported_seq":4737,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.704967+0000","last_change":"2024-09-18T01:11:30.621101+0000","last_active":"2024-09-18T01:11:30.704967+0000","last_peered":"2024-09-18T01:11:30.704967+0000","last_clean":"2024-09-18T01:11:30.704967+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:11:30.704967+0000","last_undegraded":"2024-09-18T01:11:30.704967+0000","last_fullsized":"2024-09-18T01:11:30.704967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1245,"log_dups_size":0,"ondisk_log_size":1245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:45.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082468500000000002,"stat_sum":{"num_bytes":16662571,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6665,"num_read_kb":21587,"num_write":1984,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.9","version":"255'737","reported_seq":12595,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.741168+0000","last_change":"2024-09-18T01:11:30.632829+0000","last_active":"2024-09-18T01:11:30.741168+0000","last_peered":"2024-09-18T01:11:30.741168+0000","last_clean":"2024-09-18T01:11:30.741168+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:11:30.741168+0000","last_undegraded":"2024-09-18T01:11:30.741168+0000","last_fullsized":"2024-09-18T01:11:30.741168+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:27.585447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049497799999999998,"stat_sum":{"num_bytes":22033762,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12022,"num_read_kb":43873,"num_write":1032,"num_write_kb":21817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.a","version":"254'495","reported_seq":10776,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:30.632496+0000","last_change":"2024-09-18T01:10:55.576692+0000","last_active":"2024-09-18T01:11:30.632496+0000","last_peered":"2024-09-18T01:11:30.632496+0000","last_clean":"2024-09-18T01:10:50.135462+0000","last_became_active":"2024-09-18T01:10:55.576692+0000","last_became_peered":"2024-09-18T01:10:55.576692+0000","last_unstale":"2024-09-18T01:11:30.632496+0000","last_undegraded":"2024-09-18T01:10:55.576179+0000","last_fullsized":"2024-09-18T01:10:55.576069+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:58.985297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058732200000000004,"stat_sum":{"num_bytes":42510450,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10295,"num_read_kb":42923,"num_write":614,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":25}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"255'745","reported_seq":5207,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-18T01:11:30.717041+0000","last_change":"2024-09-18T01:11:30.621666+0000","last_active":"2024-09-18T01:11:30.717041+0000","last_peered":"2024-09-18T01:11:30.717041+0000","last_clean":"2024-09-18T01:11:30.717041+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:11:30.717041+0000","last_undegraded":"2024-09-18T01:11:30.717041+0000","last_fullsized":"2024-09-18T01:11:30.717041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:44:45.759277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075392199999999997,"stat_sum":{"num_bytes":12508096,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5270,"num_read_kb":33823,"num_write":971,"num_write_kb":24417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.c","version":"256'1016","reported_seq":6958,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.643258+0000","last_change":"2024-09-18T01:11:32.643447+0000","last_active":"2024-09-18T01:11:32.643258+0000","last_peered":"2024-09-18T01:11:32.643258+0000","last_clean":"2024-09-18T01:11:32.643258+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:11:32.643258+0000","last_undegraded":"2024-09-18T01:11:32.643258+0000","last_fullsized":"2024-09-18T01:11:32.643258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:06.672587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035713999999999999,"stat_sum":{"num_bytes":11663483,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7937,"num_read_kb":32685,"num_write":1667,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.d","version":"255'679","reported_seq":14133,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.633731+0000","last_change":"2024-09-18T01:11:32.633962+0000","last_active":"2024-09-18T01:11:32.633731+0000","last_peered":"2024-09-18T01:11:32.633731+0000","last_clean":"2024-09-18T01:11:32.633731+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:11:32.633731+0000","last_undegraded":"2024-09-18T01:11:32.633731+0000","last_fullsized":"2024-09-18T01:11:32.633731+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:15.147696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079718400000000004,"stat_sum":{"num_bytes":17517309,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":13756,"num_read_kb":53109,"num_write":928,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.e","version":"257'1273","reported_seq":7804,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.717777+0000","last_change":"2024-09-18T01:11:32.634750+0000","last_active":"2024-09-18T01:11:32.717777+0000","last_peered":"2024-09-18T01:11:32.717777+0000","last_clean":"2024-09-18T01:11:32.717777+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:11:32.717777+0000","last_undegraded":"2024-09-18T01:11:32.717777+0000","last_fullsized":"2024-09-18T01:11:32.717777+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:37:49.224895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015353299999999999,"stat_sum":{"num_bytes":29868268,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2031,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.f","version":"254'645","reported_seq":6022,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.634595+0000","last_change":"2024-09-18T01:11:32.634824+0000","last_active":"2024-09-18T01:11:32.634595+0000","last_peered":"2024-09-18T01:11:32.634595+0000","last_clean":"2024-09-18T01:11:32.634595+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:11:32.634595+0000","last_undegraded":"2024-09-18T01:11:32.634595+0000","last_fullsized":"2024-09-18T01:11:32.634595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:26:21.628089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001558963,"stat_sum":{"num_bytes":9284284,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5779,"num_read_kb":31056,"num_write":901,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.10","version":"256'536","reported_seq":4266,"reported_epoch":257,"state":"active+undersized+degraded","last_fresh":"2024-09-18T01:11:32.628032+0000","last_change":"2024-09-18T01:10:55.580456+0000","last_active":"2024-09-18T01:11:32.628032+0000","last_peered":"2024-09-18T01:11:32.628032+0000","last_clean":"2024-09-18T01:10:49.782749+0000","last_became_active":"2024-09-18T01:10:55.580456+0000","last_became_peered":"2024-09-18T01:10:55.580456+0000","last_unstale":"2024-09-18T01:11:32.628032+0000","last_undegraded":"2024-09-18T01:10:55.578006+0000","last_fullsized":"2024-09-18T01:10:55.577905+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":33,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:31.066759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038286899999999998,"stat_sum":{"num_bytes":11778783,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":3854,"num_read_kb":28597,"num_write":715,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"257'636","reported_seq":4821,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.710710+0000","last_change":"2024-09-18T01:11:32.633606+0000","last_active":"2024-09-18T01:11:32.710710+0000","last_peered":"2024-09-18T01:11:32.710710+0000","last_clean":"2024-09-18T01:11:32.710710+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:11:32.710710+0000","last_undegraded":"2024-09-18T01:11:32.710710+0000","last_fullsized":"2024-09-18T01:11:32.710710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:15.474981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055697699999999995,"stat_sum":{"num_bytes":19969431,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":828,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.12","version":"257'1353","reported_seq":6678,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.754350+0000","last_change":"2024-09-18T01:11:32.630949+0000","last_active":"2024-09-18T01:11:32.754350+0000","last_peered":"2024-09-18T01:11:32.754350+0000","last_clean":"2024-09-18T01:11:32.754350+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:11:32.754350+0000","last_undegraded":"2024-09-18T01:11:32.754350+0000","last_fullsized":"2024-09-18T01:11:32.754350+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:37:53.237111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022966219999999999,"stat_sum":{"num_bytes":13862708,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8917,"num_read_kb":29491,"num_write":2240,"num_write_kb":20048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]},{"pgid":"3.13","version":"257'909","reported_seq":6038,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-18T01:11:32.754983+0000","last_change":"2024-09-18T01:11:32.633656+0000","last_active":"2024-09-18T01:11:32.754983+0000","last_peered":"2024-09-18T01:11:32.754983+0000","last_clean":"2024-09-18T01:11:32.754983+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:11:32.754983+0000","last_undegraded":"2024-09-18T01:11:32.754983+0000","last_fullsized":"2024-09-18T01:11:32.754983+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:46.372670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057219499999999995,"stat_sum":{"num_bytes":1759610,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7140,"num_read_kb":22847,"num_write":1299,"num_write_kb":16539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"1"},{"start":"26","length":"1"},{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3c","length":"1"},{"start":"3e","length":"1"},{"start":"40","length":"1"},{"start":"42","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"},{"start":"4a","length":"1"},{"start":"4c","length":"1"},{"start":"4e","length":"1"},{"start":"50","length":"1"},{"start":"52","length":"1"},{"start":"54","length":"1"},{"start":"56","length":"1"},{"start":"58","length":"1"},{"start":"5a","length":"1"},{"start":"5c","length":"1"},{"start":"5e","length":"1"},{"start":"60","length":"1"},{"start":"62","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":635517541,"num_objects":683,"num_object_clones":0,"num_object_copies":1366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":683,"num_whiteouts":0,"num_read":262375,"num_read_kb":1259582,"num_write":34939,"num_write_kb":849762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":486592512,"data_stored":484644670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24546,"ondisk_log_size":24546,"up":59,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159596,"kb_used_data":73696,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94212244,"statfs":{"total":96636764160,"available":96473337856,"internally_reserved":0,"allocated":75464704,"data_stored":73669504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87357898},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85196,"kb_used_data":45680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286644,"statfs":{"total":96636764160,"available":96549523456,"internally_reserved":0,"allocated":46776320,"data_stored":45024779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155144,"kb_used_data":79464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94216696,"statfs":{"total":96636764160,"available":96477896704,"internally_reserved":0,"allocated":81371136,"data_stored":79574210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509496,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70532,"kb_used_data":31024,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301308,"statfs":{"total":96636764160,"available":96564539392,"internally_reserved":0,"allocated":31768576,"data_stored":30092004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94340,"kb_used_data":45104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94277500,"statfs":{"total":96636764160,"available":96540160000,"internally_reserved":0,"allocated":46186496,"data_stored":44392174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134112,"kb_used_data":66828,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94237728,"statfs":{"total":96636764160,"available":96499433472,"internally_reserved":0,"allocated":68431872,"data_stored":66624231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161296,"kb_used_data":89800,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94210544,"statfs":{"total":96636764160,"available":96471597056,"internally_reserved":0,"allocated":91955200,"data_stored":90045886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134568,"kb_used_data":71432,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94237272,"statfs":{"total":96636764160,"available":96498966528,"internally_reserved":0,"allocated":73146368,"data_stored":71335510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64741376,"data_stored":64504460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42942464,"data_stored":42737229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28659712,"data_stored":28555783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69705728,"data_stored":69465406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88285184,"data_stored":87940438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":76734464,"data_stored":76516004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44171264,"data_stored":43753450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71352320,"data_stored":71171900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:11:33.812 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [1, 4, 6, 5] 2024-09-18T01:11:33.812 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.9', '1', '4', '6', '5'] 2024-09-18T01:11:33.813 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.9 1 4 6 5 2024-09-18T01:11:33.817 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:33.817 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:33.817 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.826 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:33.868 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:33.867+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:33.969 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:33.967+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:33.985 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:33.985 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:33.985 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:33.985 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:33.985 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:33.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:34.069 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:34.067+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:34.099 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437491484999995, 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.100 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:34.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437472288999999, 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.105 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:34.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437453927999996, 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:34.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.116 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437435824000005, 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:34.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:34.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437417226999997, 2024-09-18T01:11:34.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.125 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437394226999999, 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.126 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:34.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:34.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437375555000003, 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:34.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:34.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437355552, 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:34.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437337311999997, 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:34.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:34.144 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437318898000001, 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:34.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:34.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:34.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.437296871000001, 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.433045092, 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:34.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.433034640000002, 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.160 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:34.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.433022949000005, 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.165 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:34.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.169 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.433009651999996, 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:34.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:34.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.432993581000005, 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.174 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:34.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.432976922999998, 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:34.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:34.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:34.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.432957627999997, 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.188 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.432929397999999, 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:34.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:34.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:34.193 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.432908273999999, 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:34.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:34.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:34.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:34.198 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:34.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:34.200 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:34.198+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:34.300 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:34.298+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:34.310 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:34.310 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:34.319 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:34.401 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:34.399+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:34.462 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:34.462 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:34.462 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:34.470 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:34.502 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:34.501+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:34.575 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:34.575 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:34.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:34.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:34.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 216.850614439, 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:34.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819587917000007, 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:34.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:34.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819578339000003, 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:34.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:34.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:34.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819564313000001, 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.593 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:34.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:34.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819535540000004, 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:34.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:34.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819507844, 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:34.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:34.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:34.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819473396000006, 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:34.614 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819443550000003, 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:34.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:34.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819421359000003, 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:34.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:34.625 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819384452999998, 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.626 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:34.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819363338000002, 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:34.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.819332813000003, 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:34.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:34.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813852558999997, 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:34.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813798700000007, 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:34.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813721278000003, 2024-09-18T01:11:34.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:34.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:34.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813691164000005, 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:34.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:34.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813641246000003, 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.668 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:34.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813607658999999, 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:34.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:34.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813583124999994, 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:34.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:34.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.813397843000004, 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:34.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:34.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:34.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:34.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.689 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:34.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:34.690 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.9 pg_upmap_items mapping to [1->4,6->5] 2024-09-18T01:11:34.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:34.692 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:34.689+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:34.791 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:34.791+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:34.832 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:34.841 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:34.841 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:34.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:34.892 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:34.890+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:34.951 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:34.951 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:34.952 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:34.952 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:34.960 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:34.967 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:34.967 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:34.983 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:34.983 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:34.993 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:34.992+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:34.999 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:34.999 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:35.015 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:35.015 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:35.032 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:35.032 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:35.032 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:35.049 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:35.049 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:35.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:35.054 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:35.095 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:35.094+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:35.104 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.104 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.113 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:35.195 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:35.193+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:35.216 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.216 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.225 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:35.296 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:35.295+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:35.368 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.368 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.377 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:35.421 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:35.419+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:35.480 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.480 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.488 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:35.532 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:35.531+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:35.592 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.593 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.598 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:35.597+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:35.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:35.668 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:11:35.699 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:35.697+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:35.757 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.758 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:35.800 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:35.798+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:35.874 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.874 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.883 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:35.901 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:35.900+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:35.987 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:35.988 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:35.996 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:36.002 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:36.001+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:36.099 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.100 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.103 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:36.102+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:36.108 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:36.204 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:36.202+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:36.211 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.211 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:36.305 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:36.303+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:36.323 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.323 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.332 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:36.406 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:36.405+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:36.433 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.434 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.442 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:36.506 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:36.505+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:36.585 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.585 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:36.637 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:36.636+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:36.642 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:36.708 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:36.707+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:36.710 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.711 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.721 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:36.809 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:36.807+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:36.879 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.880 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:36.888 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:36.932 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:36.931+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:36.992 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:36.992 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:37.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:37.045 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:37.043+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:37.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:37.105 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:37.106 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:37.111 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:37.109+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:37.114 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:37.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:37.173 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:37.189 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:37.190 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:37.205 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:37.205 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:37.212 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:37.210+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:37.221 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:37.221 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:37.238 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:37.238 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:37.255 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:37.255 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:37.255 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:37.259 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:37.259 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:37.260 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:37.260 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:37.260 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:37.268 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:37.271 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:37.272 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:37.275 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:37.275 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:37.352 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:37.350+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:37.413 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:37.413 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 219.688829384, 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:37.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.420 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.657802862, 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.421 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:37.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65779328399999, 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:37.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65777925800001, 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.432 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65775048499999, 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.437 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:37.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:37.442 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.657722789, 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:37.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.657688341, 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65765849500001, 2024-09-18T01:11:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65763630399999, 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.657599398, 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65757828300001, 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65754775800001, 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.652067504, 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.652013645, 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65193622300001, 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.651906109, 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65185619099999, 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.651822604, 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65179807, 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.65161278799999, 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.532 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:37.532 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:37.412+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:37.533 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:37.534 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:37.531+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:37.634 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:37.633+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:37.680 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:37.680 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:37.681 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:37.681 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:37.689 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:37.735 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:37.733+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:37.801 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:37.801 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:37.809 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:37.836 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:37.835+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:37.937 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:37.936+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:37.956 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.355070212000001, 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353920196999994, 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353900886000005, 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353873140000005, 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353853287999996, 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353833424000001, 2024-09-18T01:11:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353815217999994, 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353795171000002, 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:37.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:37.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353776963000001, 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:37.997 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:37.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.002 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353758850999995, 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:38.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353740932999997, 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:38.007 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353722912999999, 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353169957000006, 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353151233999995, 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353121451000007, 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.353057363000005, 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.352873062, 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.352854907999998, 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.352826535999995, 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.352804610000007, 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.063 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:38.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:38.065 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:38.063+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:38.165 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:38.163+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:38.169 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:38.169 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:38.170 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:38.170 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:38.178 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:38.266 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:38.265+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:38.323 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:38.353 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:38.368 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:38.367+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 266.55823470500002, 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.83428415399999, 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.86677968699999, 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.86180735799999, 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.856914975, 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.851500337, 2024-09-18T01:11:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.83992607099999, 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.83095837800001, 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.830903916, 2024-09-18T01:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.82563254199999, 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.825551803, 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.824590804, 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.82453201200001, 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.82336804400001, 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.823003375, 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.82248083899999, 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.82208345799999, 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.820230728, 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.820108537, 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:258055 3.19 3:9ff6078a:::rbd_id.image_client.0-clone23:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e249)", 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.816617859000001, 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95186631499999996, 2024-09-18T01:11:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 258055 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639427+0000", 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9410000000000001e-06 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639434+0000", 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8730000000000001e-06 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639451+0000", 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6594000000000001e-05 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639458+0000", 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4789999999999999e-06 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639509+0000", 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0645e-05 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639516+0000", 2024-09-18T01:11:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1810000000000004e-06 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588228+0000", 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94871193499999995 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588377+0000", 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014855700000000001 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588500+0000", 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000123497 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.590468+0000", 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019678320000000001 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591148+0000", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068034900000000001 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591217+0000", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8204e-05 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591292+0000", 2024-09-18T01:11:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5228000000000005e-05 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:38.582 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:38.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:38.584 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:38.582+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:38.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:11:38.684 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:38.683+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:38.705 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:38.705 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:38.705 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:38.715 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:38.785 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:38.784+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:38.862 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:38.862 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:38.862 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:38.862 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:38.862 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:38.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:38.915 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:38.914+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 221.25214745400001, 2024-09-18T01:11:38.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:38.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:38.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:38.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:38.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:38.984 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.228689364, 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:38.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:38.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.215611112, 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:38.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:38.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.215590855, 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.997 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:38.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:38.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:39.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21524582000001, 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:39.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:39.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21493869, 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.008 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214902931, 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.014 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:39.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214864662, 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:39.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.025 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21484019, 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:39.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:39.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214819529, 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:39.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:39.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.036 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214800905, 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:39.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214781114, 2024-09-18T01:11:39.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:39.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:39.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:39.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214762011, 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:39.048 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:39.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21474392099999, 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.054 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:39.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:39.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21472582200001, 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.059 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:39.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.21470749300001, 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:39.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:39.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214689127, 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:39.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214670867, 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214652413, 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.214602727, 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:39.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:39.093 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:39.092+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:39.194 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:39.193+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:39.236 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:39.236 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:39.237 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:39.237 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:39.245 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:39.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:39.295 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:39.294+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:39.350 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:39.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:39.350 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:39.350 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:39.350 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:39.359 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:39.403 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:39.402+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:39.411 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:39.411 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:39.428 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:39.428 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:39.444 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:39.444 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:39.461 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:39.461 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:39.466 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:39.466 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:39.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:39.478 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:39.478 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:39.496 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:39.496 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:39.495+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:39.513 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:39.513 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:39.517 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:39.517 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:39.583 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:39.583 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:39.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:39.637 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:39.636+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:39.666 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-18T01:11:39.694 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:39.694 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:39.694 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T01:11:39.694 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:11:39.694 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:11:39.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T01:11:39.701 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:39.701 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:39.701 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:39.700+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:39.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:39.798 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:39.797+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:39.806 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:39.806 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "" 2024-09-18T01:11:39.806 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:39.856 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:39.856 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:39.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:39.898 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:39.897+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:39.981 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:39.981 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:39.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:39.999 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:39.998+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:40.093 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.093 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.099 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:40.099+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:40.102 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:40.200 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:40.199+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:40.245 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.245 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:40.300 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:40.300+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:40.361 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.361 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:40.401 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:40.400+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:40.474 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.474 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.483 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:40.502 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:40.501+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:40.586 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.586 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:40.603 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:40.602+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:40.703 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.703 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.704 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:40.702+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:40.713 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:40.804 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:40.803+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:40.861 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:40.861 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:40.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:40.905 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:40.904+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:41.005 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:41.004+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:41.014 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:41.014 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:41.023 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:41.107 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:41.106+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:41.168 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:41.169 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:41.178 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:41.206 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:41.205+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:41.307 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:41.306+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:41.322 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:41.322 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:41.331 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:41.408 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:41.407+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:41.474 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:41.474 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:41.483 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:41.518 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:41.527 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:41.526+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:41.599 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:41.600 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:41.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:41.610 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:41.608+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:41.655 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:41.655 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:41.672 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:41.672 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:41.684 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:11:41.697 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:41.697 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:41.710 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:41.709+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:41.716 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:41.716 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:41.724 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:41.725 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:41.725 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:41.725 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:41.725 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:41.736 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:41.736 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:41.736 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:41.754 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:41.755 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:41.755 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:41.781 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:41.781 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:41.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:41.787 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:41.828 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:41.827+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:28508 3.10 3:0d139f4b:::rbd_data.1176e641238f.0000000000000008:head [write 241271~4489] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 224.16022427600001, 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.92002805799999998, 2024-09-18T01:11:41.886 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 28508 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722680+0000", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722681+0000", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0049999999999999e-06 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722690+0000", 2024-09-18T01:11:41.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1200000000000002e-06 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722700+0000", 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0764000000000001e-05 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722706+0000", 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8309999999999997e-06 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722779+0000", 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2794999999999997e-05 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:57.722790+0000", 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1323e-05 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.634962+0000", 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.912171591 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635091+0000", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012921800000000001 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.635186+0000", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5067000000000003e-05 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.636183+0000", 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099649200000000008 2024-09-18T01:11:41.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642639+0000", 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006455896 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642677+0000", 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8424999999999999e-05 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:07:58.642709+0000", 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1531e-05 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:185713 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 229376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.136766186, 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84592358599999995, 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 185713 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746139+0000", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746141+0000", 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1299999999999999e-06 2024-09-18T01:11:41.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746147+0000", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9510000000000003e-06 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.746152+0000", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6450000000000004e-06 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747152+0000", 2024-09-18T01:11:41.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001000314 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.747162+0000", 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0434e-05 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591875+0000", 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84471286099999998 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.591954+0000", 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8586999999999998e-05 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.592062+0000", 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108336 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186337 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 239616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:41.898 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.123687934, 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84606753700000004, 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186337 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759217+0000", 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759220+0000", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2620000000000001e-06 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759226+0000", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3690000000000003e-06 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759231+0000", 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7650000000000001e-06 2024-09-18T01:11:41.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761263+0000", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002032701 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761271+0000", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5989999999999996e-06 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605161+0000", 2024-09-18T01:11:41.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84389028899999996 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605213+0000", 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1635000000000003e-05 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605284+0000", 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1499000000000001e-05 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186338 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 245760~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.123667677, 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84628621400000004, 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186338 2024-09-18T01:11:41.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759237+0000", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759240+0000", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2139999999999999e-06 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759246+0000", 2024-09-18T01:11:41.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7039999999999997e-06 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759251+0000", 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6920000000000001e-06 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761302+0000", 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020513380000000002 2024-09-18T01:11:41.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761322+0000", 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9349000000000001e-05 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605335+0000", 2024-09-18T01:11:41.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84401364000000001 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605418+0000", 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2781000000000006e-05 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605523+0000", 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000105078 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186470 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12332264200001, 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609458900000001, 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186470 2024-09-18T01:11:41.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759582+0000", 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759585+0000", 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.074e-06 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759590+0000", 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9190000000000004e-06 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759595+0000", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761350+0000", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00175475 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761358+0000", 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:11:41.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605562+0000", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84420431699999998 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605611+0000", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605677+0000", 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5827000000000005e-05 2024-09-18T01:11:41.914 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186471 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 62464~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12301551199999, 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.846019034, 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.915 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186471 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759889+0000", 2024-09-18T01:11:41.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759890+0000", 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2799999999999996e-07 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759894+0000", 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6899999999999998e-06 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759905+0000", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.117e-05 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759912+0000", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2089999999999999e-06 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761385+0000", 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014730310000000001 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761393+0000", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6629999999999993e-06 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605713+0000", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84432022100000004 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605801+0000", 2024-09-18T01:11:41.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8442000000000005e-05 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605908+0000", 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010698 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186472 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 68608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:41.920 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122979753, 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84615272200000002, 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186472 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759925+0000", 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759926+0000", 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4199999999999996e-07 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759931+0000", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7649999999999999e-06 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759942+0000", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.093e-05 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759948+0000", 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0249999999999999e-06 2024-09-18T01:11:41.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761423+0000", 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001474508 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761430+0000", 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7580000000000003e-06 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605948+0000", 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84451866799999997 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.605997+0000", 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9128999999999997e-05 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606078+0000", 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0396999999999996e-05 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.925 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186473 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12294148399999, 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84633966500000002, 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186473 2024-09-18T01:11:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759963+0000", 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759964+0000", 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759967+0000", 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7379999999999999e-06 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759976+0000", 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759981+0000", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761476+0000", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001494595 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761485+0000", 2024-09-18T01:11:41.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7589999999999992e-06 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606103+0000", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84461787099999996 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606181+0000", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8164999999999994e-05 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606303+0000", 2024-09-18T01:11:41.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012152800000000001 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186474 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 113664~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122917012, 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84648789999999996, 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.931 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186474 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759988+0000", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759990+0000", 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1600000000000001e-06 2024-09-18T01:11:41.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.759997+0000", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.131e-06 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760003+0000", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2249999999999999e-06 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761517+0000", 2024-09-18T01:11:41.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015142459999999999 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761524+0000", 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6179999999999997e-06 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606340+0000", 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84481519800000004 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606403+0000", 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2812999999999996e-05 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606476+0000", 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3133999999999995e-05 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186475 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 119808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:41.936 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12289635099999, 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84684653200000004, 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186475 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760008+0000", 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760009+0000", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.39e-07 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760011+0000", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0490000000000002e-06 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760017+0000", 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0480000000000004e-06 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760022+0000", 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.843e-06 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761558+0000", 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536117 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761566+0000", 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8399999999999995e-06 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606698+0000", 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84513284 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606781+0000", 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2538000000000002e-05 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606855+0000", 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3918000000000006e-05 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186476 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122877727, 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84710338600000001, 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186476 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760027+0000", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760030+0000", 2024-09-18T01:11:41.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1799999999999999e-06 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760036+0000", 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9839999999999998e-06 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760041+0000", 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9749999999999998e-06 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761612+0000", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001571268 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761620+0000", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1640000000000001e-06 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.606897+0000", 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84527666300000004 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607020+0000", 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012380799999999999 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607130+0000", 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109992 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186477 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 132096~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122857936, 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84735291499999998, 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186477 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760047+0000", 2024-09-18T01:11:41.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760049+0000", 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760055+0000", 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9309999999999996e-06 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760060+0000", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000003e-06 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761648+0000", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015880270000000001 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761656+0000", 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:41.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607175+0000", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84551907699999995 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607278+0000", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103448 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607400+0000", 2024-09-18T01:11:41.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121574 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186478 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 171008~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122838833, 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84754911700000002, 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.953 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186478 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760066+0000", 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.46e-07 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760068+0000", 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9190000000000002e-06 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760074+0000", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7339999999999996e-06 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760079+0000", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6659999999999999e-06 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761686+0000", 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016076459999999999 2024-09-18T01:11:41.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761696+0000", 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6840000000000008e-06 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607432+0000", 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84573583600000002 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607508+0000", 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6039999999999997e-05 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607615+0000", 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107246 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186479 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 177152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.958 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12282074300001, 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84776400399999996, 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186479 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760084+0000", 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760086+0000", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0499999999999999e-06 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760092+0000", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760097+0000", 2024-09-18T01:11:41.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6330000000000004e-06 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761723+0000", 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016261660000000001 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761735+0000", 2024-09-18T01:11:41.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.208e-05 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607643+0000", 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84590825400000003 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607747+0000", 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010361600000000001 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607848+0000", 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010105799999999999 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186480 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 183296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122802644, 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84793391100000004, 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186480 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760102+0000", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760105+0000", 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:11:41.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760110+0000", 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760115+0000", 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761772+0000", 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001656535 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761780+0000", 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607874+0000", 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84609436800000004 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.607947+0000", 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2254000000000003e-05 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608036+0000", 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9438000000000007e-05 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186481 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 189440~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122784315, 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84809710800000004, 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186481 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760120+0000", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760121+0000", 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.89e-07 2024-09-18T01:11:41.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760123+0000", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760129+0000", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760133+0000", 2024-09-18T01:11:41.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761823+0000", 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001689659 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761831+0000", 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9659999999999996e-06 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608062+0000", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84623108599999997 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608132+0000", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-05 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608218+0000", 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5545999999999997e-05 2024-09-18T01:11:41.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186482 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 228352~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122765949, 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84827439100000002, 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.975 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186482 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760139+0000", 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760141+0000", 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0279999999999999e-06 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760147+0000", 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7409999999999998e-06 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760152+0000", 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8199999999999996e-06 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761858+0000", 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001706585 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761866+0000", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.559e-06 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608253+0000", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84638716300000005 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608319+0000", 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6363000000000001e-05 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608413+0000", 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3776000000000005e-05 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186483 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 234496~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12274768899999, 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84850592199999997, 2024-09-18T01:11:41.980 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186483 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760157+0000", 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5900000000000003e-07 2024-09-18T01:11:41.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760159+0000", 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9420000000000002e-06 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760165+0000", 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6790000000000002e-06 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760170+0000", 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8520000000000003e-06 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761894+0000", 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001724267 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761902+0000", 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9060000000000002e-06 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608457+0000", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84655501200000005 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608530+0000", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2801999999999999e-05 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608663+0000", 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000133103 2024-09-18T01:11:41.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186484 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 240640~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.12272923499999, 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84873480499999998, 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.986 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186484 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760176+0000", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760181+0000", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.412e-06 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760189+0000", 2024-09-18T01:11:41.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000003e-06 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760194+0000", 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8709999999999996e-06 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761930+0000", 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001736139 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761937+0000", 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3509999999999996e-06 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608700+0000", 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84676262599999996 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608800+0000", 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100222 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608910+0000", 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011031699999999999 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:186485 3.1 3:84a626b9:::rbd_data.11762ddd5718.0000000000000021:48 [read 246784~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.122679549, 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.8489681, 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:41.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 186485 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760225+0000", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760226+0000", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8999999999999997e-07 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760232+0000", 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8289999999999996e-06 2024-09-18T01:11:41.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760253+0000", 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1392000000000001e-05 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.760267+0000", 2024-09-18T01:11:41.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4331999999999999e-05 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761971+0000", 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017034579999999999 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:56.761984+0000", 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3522e-05 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.608963+0000", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.84697887199999999 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609049+0000", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6372000000000006e-05 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:57.609193+0000", 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000143833 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:41.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:41.998 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:41.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:42.000 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:41.998+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:42.100 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:42.099+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:42.143 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:42.144 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:42.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:42.236 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:42.235+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:42.295 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.295 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:42.295 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:42.295 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:42.295 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:42.303 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:42.302+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:42.304 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:42.405 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:42.404+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.450 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194177 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 247808~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.53654532900001, 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068597230999999995, 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194177 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912041+0000", 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912042+0000", 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4999999999999998e-07 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912043+0000", 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912049+0000", 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7150000000000003e-06 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912237+0000", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018835499999999999 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.979998+0000", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067760514999999993 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980074+0000", 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6515999999999997e-05 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980638+0000", 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056393000000000003 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194178 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 253952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.53633729800001, 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068717361000000005, 2024-09-18T01:11:42.455 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194178 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912249+0000", 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912250+0000", 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8899999999999999e-07 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912257+0000", 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912265+0000", 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0569999999999993e-06 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.912270+0000", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1769999999999997e-06 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980667+0000", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068397217999999996 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980746+0000", 2024-09-18T01:11:42.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.980967+0000", 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022048799999999999 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194518 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:42.460 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522300723, 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068475765999999993, 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194518 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926286+0000", 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926289+0000", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3309999999999998e-06 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926296+0000", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0140000000000001e-06 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926301+0000", 2024-09-18T01:11:42.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994388+0000", 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068086625999999997 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994471+0000", 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3463999999999999e-05 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994762+0000", 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029008000000000001 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194519 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 141312~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522277375, 2024-09-18T01:11:42.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068738172, 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194519 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:42.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926309+0000", 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926310+0000", 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926312+0000", 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926319+0000", 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8349999999999998e-06 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926325+0000", 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994781+0000", 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068456613999999999 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.994828+0000", 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6965999999999997e-05 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995047+0000", 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021916399999999999 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194520 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.52225585399999, 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069064344999999999, 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.470 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194520 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926331+0000", 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926333+0000", 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9609999999999999e-06 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926340+0000", 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6150000000000002e-06 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926345+0000", 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4330000000000003e-06 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995089+0000", 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068743993000000003 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995158+0000", 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8934000000000001e-05 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995395+0000", 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023703799999999999 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194521 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522235135, 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069389897000000006, 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194521 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926351+0000", 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926352+0000", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6599999999999996e-07 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926354+0000", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-06 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926360+0000", 2024-09-18T01:11:42.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926366+0000", 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1839999999999998e-06 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995432+0000", 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069065953999999999 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995487+0000", 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5232999999999998e-05 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995741+0000", 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025441500000000002 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194522 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 192512~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522214359, 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069703493000000005, 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194522 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926372+0000", 2024-09-18T01:11:42.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.15e-07 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926375+0000", 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.181e-06 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926381+0000", 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4749999999999998e-06 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926386+0000", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995772+0000", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069385991999999994 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.995858+0000", 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5422999999999998e-05 2024-09-18T01:11:42.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996076+0000", 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021802599999999999 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194523 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 198656~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:42.484 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522194214, 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069943543999999996, 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194523 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926392+0000", 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926393+0000", 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6100000000000001e-07 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926395+0000", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8840000000000001e-06 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926401+0000", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926407+0000", 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5570000000000003e-06 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996118+0000", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069711182999999996 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996200+0000", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1887999999999999e-05 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996336+0000", 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136086 2024-09-18T01:11:42.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194524 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 204800~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522173838, 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070205406999999997, 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194524 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:42.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926413+0000", 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4299999999999999e-07 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926415+0000", 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0449999999999999e-06 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926422+0000", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5030000000000002e-06 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926427+0000", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1889999999999998e-06 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996372+0000", 2024-09-18T01:11:42.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.069945587000000004 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996435+0000", 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3072999999999997e-05 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996618+0000", 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018266699999999999 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194525 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 243712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.52215311, 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070501109000000006, 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194525 2024-09-18T01:11:42.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926433+0000", 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926434+0000", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926436+0000", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2309999999999999e-06 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926443+0000", 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:11:42.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926451+0000", 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996652+0000", 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070200950999999998 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996740+0000", 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7447000000000005e-05 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996934+0000", 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019498700000000001 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194526 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 249856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.52212865200001, 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070786055000000001, 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194526 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926458+0000", 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926460+0000", 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.125e-06 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926467+0000", 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6340000000000003e-06 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926472+0000", 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.78e-06 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.996969+0000", 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070497147999999996 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997052+0000", 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3354000000000002e-05 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997244+0000", 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000191609 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194527 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 256000~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.522108175, 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071068615000000002, 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194527 2024-09-18T01:11:42.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926478+0000", 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926479+0000", 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926487+0000", 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3899999999999993e-06 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926494+0000", 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.926500+0000", 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6010000000000002e-06 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997276+0000", 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.070776849000000003 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997362+0000", 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5830999999999999e-05 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.997547+0000", 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018465699999999999 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194721 3.4 3:25696e5b:::rbd_data.11762ddd5718.0000000000000053:48 [read 148480~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.513331979, 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068890949000000007, 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194721 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.509 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935255+0000", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935258+0000", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935264+0000", 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4500000000000001e-06 2024-09-18T01:11:42.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935269+0000", 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.04e-06 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002551+0000", 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.067281461000000001 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:42.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.002633+0000", 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2313999999999998e-05 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004145+0000", 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015125970000000001 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.513 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194722 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.51330697900001, 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068907655999999998, 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194722 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935280+0000", 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:42.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935283+0000", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5849999999999998e-06 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935289+0000", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5640000000000002e-06 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935308+0000", 2024-09-18T01:11:42.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9397000000000001e-05 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004177+0000", 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068868214999999997 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004187+0000", 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0485999999999999e-05 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194723 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.51326982400001, 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068905702999999999, 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194723 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935317+0000", 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935325+0000", 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6000000000000001e-06 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935332+0000", 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935337+0000", 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004214+0000", 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068877011000000002 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004222+0000", 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6500000000000002e-06 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194724 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.513243141, 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068917464999999997, 2024-09-18T01:11:42.522 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194724 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935343+0000", 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935344+0000", 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935346+0000", 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2979999999999999e-06 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935352+0000", 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0900000000000001e-06 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935357+0000", 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.921e-06 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004251+0000", 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068893902000000007 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004261+0000", 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8279999999999998e-06 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194725 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.513223242, 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068935596000000002, 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194725 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.527 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935363+0000", 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935364+0000", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7500000000000001e-07 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935366+0000", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9350000000000001e-06 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935371+0000", 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8150000000000002e-06 2024-09-18T01:11:42.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935376+0000", 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004290+0000", 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068913673999999994 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004299+0000", 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2760000000000006e-06 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194726 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.513204035, 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068953650000000005, 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194726 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935382+0000", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935383+0000", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2e-07 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935385+0000", 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9589999999999998e-06 2024-09-18T01:11:42.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935391+0000", 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9649999999999996e-06 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935396+0000", 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004326+0000", 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068930693000000001 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004336+0000", 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8130000000000007e-06 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.535 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194727 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.513185022, 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068969516999999994, 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194727 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935401+0000", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935402+0000", 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-09-18T01:11:42.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935404+0000", 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0040000000000002e-06 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935410+0000", 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.716e-06 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935414+0000", 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5709999999999999e-06 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004362+0000", 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068948152999999998 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004371+0000", 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7090000000000008e-06 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.539 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:194728 3.4 3.da7696a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:42.540 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 103.51316674900001, 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068986705999999995, 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 194728 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935420+0000", 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.84e-07 2024-09-18T01:11:42.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935422+0000", 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.852e-06 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935435+0000", 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3196999999999999e-05 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:58.935440+0000", 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.062e-06 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004395+0000", 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.068955248999999996 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:59.004406+0000", 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0961999999999999e-05 2024-09-18T01:11:42.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.545 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.545 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.545 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:42.546 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:42.547 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:42.545+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:42.647 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:42.646+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:42.685 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-18T01:11:42.695 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:42.695 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:42.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:42.747 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:42.746+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:42.848 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:42.847+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:42.859 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.859 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:42.859 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:42.859 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:42.859 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:42.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:42.949 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:42.948+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:42.972 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123695 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30995377400001, 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90219444900000001, 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123695 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660814+0000", 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660816+0000", 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660822+0000", 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660835+0000", 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3001e-05 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660844+0000", 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3549999999999994e-06 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660910+0000", 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5648999999999994e-05 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660918+0000", 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2330000000000006e-06 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.562992+0000", 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90207358900000001 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563008+0000", 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6668e-05 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123707 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30989954899999, 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90217865399999997, 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123707 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.979 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660868+0000", 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660869+0000", 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8100000000000003e-07 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660871+0000", 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7499999999999999e-06 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660878+0000", 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660884+0000", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660943+0000", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8959000000000003e-05 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660948+0000", 2024-09-18T01:11:42.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6990000000000002e-06 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563039+0000", 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209173399999998 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563047+0000", 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4590000000000001e-06 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123719 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.309867091, 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90218242400000004, 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123719 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660901+0000", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660905+0000", 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5710000000000002e-06 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660911+0000", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8560000000000001e-06 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660917+0000", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0860000000000001e-06 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660973+0000", 2024-09-18T01:11:42.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6876999999999998e-05 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.660978+0000", 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563074+0000", 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90209595200000003 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563083+0000", 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.524e-06 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123731 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30973582799999, 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.902085209, 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123731 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661032+0000", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661033+0000", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-06 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661038+0000", 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9230000000000001e-06 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661055+0000", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6807999999999999e-05 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661064+0000", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661113+0000", 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:11:42.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661120+0000", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5860000000000001e-06 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563109+0000", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90198933400000003 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563117+0000", 2024-09-18T01:11:42.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.011e-06 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123743 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.309265842, 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90166178399999997, 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:42.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123743 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:42.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661502+0000", 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661503+0000", 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0979999999999999e-06 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661508+0000", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6519999999999997e-06 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661522+0000", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4100999999999999e-05 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661532+0000", 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.047e-05 2024-09-18T01:11:42.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661585+0000", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2655000000000003e-05 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.661592+0000", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3640000000000001e-06 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563152+0000", 2024-09-18T01:11:42.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90155994399999995 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563164+0000", 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.15e-05 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123755 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30810324199999, 2024-09-18T01:11:42.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90053843600000005, 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123755 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:43.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662664+0000", 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662666+0000", 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.28e-06 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662673+0000", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5059999999999998e-06 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662690+0000", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6648000000000001e-05 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662698+0000", 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1249999999999993e-06 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662755+0000", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662762+0000", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8689999999999998e-06 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563192+0000", 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90043029799999996 2024-09-18T01:11:43.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563203+0000", 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0784e-05 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123767 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30799596, 2024-09-18T01:11:43.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90047105800000005, 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123767 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:43.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662772+0000", 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0500000000000004e-07 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662775+0000", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0809999999999998e-06 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662782+0000", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662787+0000", 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.113e-06 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662836+0000", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9604999999999997e-05 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662842+0000", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6570000000000002e-06 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563232+0000", 2024-09-18T01:11:43.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90039036500000003 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563243+0000", 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0258000000000001e-05 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123779 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:43.009 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30791430100001, 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90042506, 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123779 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662853+0000", 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662854+0000", 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662857+0000", 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662863+0000", 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:11:43.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662868+0000", 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7110000000000003e-06 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662907+0000", 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8661000000000001e-05 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.662912+0000", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1560000000000003e-06 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563269+0000", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90035699199999997 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563278+0000", 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6579999999999997e-06 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.014 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123791 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30739891299999, 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89995011999999996, 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.015 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123791 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663369+0000", 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663370+0000", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7199999999999997e-07 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663374+0000", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7090000000000001e-06 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663389+0000", 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839e-05 2024-09-18T01:11:43.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663398+0000", 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.354e-06 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663446+0000", 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8446000000000003e-05 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.663452+0000", 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7250000000000002e-06 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563309+0000", 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89985728300000001 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563319+0000", 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7920000000000004e-06 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.019 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:123803 3.9 3.5a797169 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 129.30657967499999, 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89916901900000001, 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.020 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 123803 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664188+0000", 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664189+0000", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7499999999999999e-07 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664195+0000", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664211+0000", 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5520999999999999e-05 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664219+0000", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000006e-06 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664273+0000", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3542000000000003e-05 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:33.664280+0000", 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5170000000000002e-06 2024-09-18T01:11:43.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563348+0000", 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.89906760900000005 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:34.563357+0000", 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3449999999999995e-06 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:144056 3.9 3:95b52144:::rbd_data.117650babc70.000000000000005a:head [set-alloc-hint object_size 524288 write_size 524288,write 83968~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 123.733314589, 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079182369999999998, 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 144056 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237453+0000", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237454+0000", 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3340000000000001e-06 2024-09-18T01:11:43.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237464+0000", 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9939999999999995e-06 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237475+0000", 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1043000000000001e-05 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237481+0000", 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3920000000000002e-06 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237531+0000", 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0683999999999998e-05 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237620+0000", 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8769000000000003e-05 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.237708+0000", 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7343999999999996e-05 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.238327+0000", 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061925200000000004 2024-09-18T01:11:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245275+0000", 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069478839999999997 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245328+0000", 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2882e-05 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:39.245371+0000", 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3659000000000003e-05 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158948 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.56645569699999, 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0097163010000000001, 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.031 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158948 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404312+0000", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.04e-07 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404314+0000", 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.39e-06 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404317+0000", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0709999999999999e-06 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.404319+0000", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410174+0000", 2024-09-18T01:11:43.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058546470000000001 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410260+0000", 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5987000000000002e-05 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.410552+0000", 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029158500000000002 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.411625+0000", 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010738290000000001 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413894+0000", 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022682309999999999 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.413991+0000", 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7219000000000005e-05 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.414028+0000", 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7457999999999999e-05 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158968 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.53861030900001, 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074875330000000002, 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158968 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432157+0000", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432158+0000", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432160+0000", 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4839999999999998e-06 2024-09-18T01:11:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432166+0000", 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2530000000000001e-06 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432171+0000", 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9779999999999996e-06 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434219+0000", 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020473850000000001 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434300+0000", 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0805999999999998e-05 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.434680+0000", 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000380894 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437769+0000", 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003088744 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439576+0000", 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001807026 2024-09-18T01:11:43.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439614+0000", 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7883999999999998e-05 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439645+0000", 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0716000000000002e-05 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158969 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.53859000999999, 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0086151639999999998, 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158969 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.044 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432178+0000", 2024-09-18T01:11:43.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432181+0000", 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.576e-06 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432187+0000", 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1210000000000003e-06 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432192+0000", 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.967e-06 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435315+0000", 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031232780000000002 2024-09-18T01:11:43.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435380+0000", 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4979000000000002e-05 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435596+0000", 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000216536 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439703+0000", 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041068140000000003 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440683+0000", 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098004199999999998 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440751+0000", 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8190999999999998e-05 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440793+0000", 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1319999999999997e-05 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158970 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.538569747, 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089861379999999994, 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158970 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432198+0000", 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.77e-07 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432201+0000", 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.548e-06 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432206+0000", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6659999999999996e-06 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432211+0000", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8300000000000003e-06 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.435994+0000", 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003782246 2024-09-18T01:11:43.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436041+0000", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7318999999999999e-05 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436235+0000", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019408699999999999 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439762+0000", 2024-09-18T01:11:43.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035269559999999999 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441095+0000", 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001332979 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441140+0000", 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5382e-05 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441184+0000", 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3748000000000002e-05 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158971 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.538549894, 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090642269999999994, 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158971 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432218+0000", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432221+0000", 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4710000000000001e-06 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432227+0000", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9580000000000004e-06 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432231+0000", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7589999999999997e-06 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436636+0000", 2024-09-18T01:11:43.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044045780000000001 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436701+0000", 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4881000000000004e-05 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.436925+0000", 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022388900000000001 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.439965+0000", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003040587 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441196+0000", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00123036 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441243+0000", 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7457000000000003e-05 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441282+0000", 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8918999999999998e-05 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:158972 3.9 3:92036f1a:::rbd_object_map.117665245fc1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 120.538530737, 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0091340629999999996, 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 158972 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432237+0000", 2024-09-18T01:11:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432240+0000", 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4269999999999998e-06 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432246+0000", 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9070000000000004e-06 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.432251+0000", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9930000000000003e-06 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437235+0000", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049846049999999996 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437286+0000", 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0871000000000001e-05 2024-09-18T01:11:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.437480+0000", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019380600000000001 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.440019+0000", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002539164 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441292+0000", 2024-09-18T01:11:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001272741 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441334+0000", 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2533000000000001e-05 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:42.441371+0000", 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6640000000000002e-05 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:160255 3.9 3:9121d0fb:::rbd_data.1176f0c0f620.000000000000000a:head [write 0~1048576 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.15454031100001, 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010126619, 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 160255 2024-09-18T01:11:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816227+0000", 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816230+0000", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.481e-06 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816723+0000", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049363800000000002 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816752+0000", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8974000000000001e-05 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816767+0000", 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4844999999999999e-05 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816837+0000", 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9528000000000002e-05 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.816984+0000", 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014761 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.817138+0000", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015310899999999999 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.819641+0000", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025038560000000001 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826226+0000", 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065841329999999998 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826320+0000", 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.826354+0000", 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3986999999999998e-05 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:161191 3.9 3:9177825b:::rbd_data.1176f0c0f620.0000000000000023:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 119.094581448, 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0090229149999999994, 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 161191 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876186+0000", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876188+0000", 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1069999999999999e-06 2024-09-18T01:11:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876328+0000", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139794 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876356+0000", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8005999999999999e-05 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876367+0000", 2024-09-18T01:11:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1058e-05 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.876869+0000", 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050200099999999997 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877188+0000", 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00031918300000000001 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.877329+0000", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014031300000000001 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.879418+0000", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020891659999999999 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885025+0000", 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056074920000000004 2024-09-18T01:11:43.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885158+0000", 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132946 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:09:43.885209+0000", 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0849000000000001e-05 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4470.0:232398 3.4 e177/143 3:270c054c:::rbd_data.11767f96bf45.0000000000000000:head v 177'314, mlcod=175'312)", 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.310078001999997, 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.010278087, 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.660690+0000", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0900000000000001e-07 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661552+0000", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086221699999999997 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661575+0000", 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2705e-05 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661584+0000", 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0159999999999997e-06 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661646+0000", 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1752000000000001e-05 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.661670+0000", 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4572000000000001e-05 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670906+0000", 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092363470000000007 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:11.670968+0000", 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1268999999999999e-05 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.086 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:43.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:43.087 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:43.086+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:43.188 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:43.187+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:43.229 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:43.230 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:43.230 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:43.230 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:43.238 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:43.289 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:43.288+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:43.341 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:43.341 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:43.341 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:43.341 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:43.341 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:43.350 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:43.389 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:43.388+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791971493999995, 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791952297999998, 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791933936999996, 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791915833000004, 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:43.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791897235999997, 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:43.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:43.478 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791874235999998, 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791855564000002, 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:43.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:43.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:43.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:43.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:43.488 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791835560999999, 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:43.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:43.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:43.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791817320999996, 2024-09-18T01:11:43.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:43.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:43.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.791798907, 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:43.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:43.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.502 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.79177688, 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.503 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:43.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:43.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:43.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787525101, 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.508 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:43.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787514649000002, 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.513 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:43.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787502958000005, 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:43.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:43.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787489660999995, 2024-09-18T01:11:43.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:43.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:43.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:43.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.527 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787473590000005, 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.528 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:43.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:43.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787456931999998, 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:43.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:43.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787437636999996, 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:43.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.544 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787409406999998, 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.545 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:43.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.549 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.787388282999999, 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:43.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:43.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:43.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:43.556 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:43.558 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:43.559 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:43.556+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:43.658 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:43.657+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:43.666 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:43.666 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:43.667 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:43.667 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:43.675 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:43.759 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:43.758+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:43.785 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:43.785 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:43.785 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:43.786 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:43.786 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:43.788 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:43.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:43.879 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:43.878+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:43.907 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:43.908 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:43.924 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:43.940 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:43.941 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:43.941 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:43.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 226.216818875, 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:43.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:43.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:43.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.957 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:43.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.185792353, 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:43.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:43.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18578277500001, 2024-09-18T01:11:43.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:43.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:43.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:43.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.185768749, 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:43.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18573997599999, 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.982 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:43.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18571228, 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:43.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.992 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.185677832, 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:43.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:43.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18564798600001, 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:43.998 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:43.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:44.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:44.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18562579500001, 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:44.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.185588889, 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.009 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:44.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:44.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18556777400001, 2024-09-18T01:11:44.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:44.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.18553724900001, 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:44.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:44.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.180056995, 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:44.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.180003136, 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:44.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.17992571400001, 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:44.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.17989559999999, 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:44.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:44.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.17984568200001, 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:44.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.179812095, 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:44.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:44.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.179787561, 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.058 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:44.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 107.17960227899999, 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:44.063 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:44.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:44.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:44.069 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:44.069 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:44.070 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:44.070 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:44.071 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:44.071 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:44.071 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:44.069+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:44.072 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:44.072 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:44.072 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:44.072 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:44.073 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:44.073 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:44.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:44.171 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:44.170+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:44.215 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.215 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.224 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:44.271 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:44.271+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:44.327 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.327 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.336 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:44.373 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:44.372+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:44.439 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.439 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:44.473 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:44.473+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:44.551 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.551 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.559 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:44.603 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:44.602+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:44.663 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.663 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:44.676 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:44.675+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:44.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:11:44.776 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:44.775+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:44.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T01:11:44.816 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:11:44.816 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-18T01:11:44.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-18T01:11:44.826 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.826 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.836 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:44.877 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:44.876+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:44.954 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:44.955 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:44.958 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:44.958 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:44.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-18T01:11:44.969 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:44.970 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:44.979 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:44.980 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:44.978+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:45.080 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:45.078+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:45.091 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:45.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:45.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:45.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:45.092 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:45.092 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:45.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:45.095 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.095 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-18T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:45.219 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:45.218+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:45.238 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:45.238 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.238 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-18T01:11:45.282 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:45.281+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:45.286 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.286 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.296 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:45.383 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:45.382+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:45.412 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:45.412 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:45.412 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:45.412 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:45.412 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:45.413 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:45.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:45.415 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-18T01:11:45.442 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.442 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.452 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:45.493 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:45.492+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:45.577 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:45.578 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:45.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:45.580 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.580 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-18T01:11:45.585 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:45.584+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:45.599 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.599 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:45.686 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:45.685+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:45.687 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:45.691 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:45.691 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.691 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-18T01:11:45.700 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:11:45.764 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.765 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:45.818 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:45.817+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:45.853 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:45.853 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:45.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-18T01:11:45.886 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:45.886 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:45.887 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:45.886+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:45.895 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:45.989 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:45.988+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:0 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:1 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:2 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:abort 2024-09-18T01:11:46.020 INFO:teuthology.orchestra.run.smithi120.stderr:assert 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:bench [] [] [] [] 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs files list 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:bluefs stats 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:bluestore allocator dump block 2024-09-18T01:11:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2024-09-18T01:11:46.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:46.023 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:11:46.043 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:46.043 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:46.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:46.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:11:46.089 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:46.088+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:46.190 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:46.190+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:46.198 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:46.198 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:46.207 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:46.213 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:46.232 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:46.232 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:46.248 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:46.249 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:46.266 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:46.266 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:46.285 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:46.291 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:46.290+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:46.303 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:46.320 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:46.320 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:46.320 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:46.320 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:46.321 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:46.321 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:46.324 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:46.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:46.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:46.414 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:46.413+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:46.472 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:46.472 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:46.472 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-09-18T01:11:46.472 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-18T01:11:46.472 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:46.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:46.522 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:46.521+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:46.594 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:46.593+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.625 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225137 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 112640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.963093375, 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072168212999999995, 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225137 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660472+0000", 2024-09-18T01:11:46.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660474+0000", 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.091e-06 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660480+0000", 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8209999999999998e-06 2024-09-18T01:11:46.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660485+0000", 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732325+0000", 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071840671999999994 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732400+0000", 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4271999999999996e-05 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732640+0000", 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024032 2024-09-18T01:11:46.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225138 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.963074179, 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072433346999999995, 2024-09-18T01:11:46.631 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225138 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660491+0000", 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660493+0000", 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1390000000000002e-06 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660499+0000", 2024-09-18T01:11:46.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.75e-06 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660504+0000", 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4739999999999999e-06 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732669+0000", 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072164995999999995 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732748+0000", 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8881000000000006e-05 2024-09-18T01:11:46.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732924+0000", 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000176725 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225139 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 140288~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.963055818, 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072765542000000002, 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225139 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660509+0000", 2024-09-18T01:11:46.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660510+0000", 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7399999999999999e-07 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660512+0000", 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660517+0000", 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8490000000000002e-06 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660522+0000", 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2230000000000001e-06 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.732953+0000", 2024-09-18T01:11:46.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072431836999999999 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733025+0000", 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1174000000000007e-05 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733275+0000", 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025012999999999999 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225140 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 154624~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.963037714, 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073096098999999998, 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225140 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.646 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660527+0000", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660528+0000", 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9900000000000002e-07 2024-09-18T01:11:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660530+0000", 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660536+0000", 2024-09-18T01:11:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2110000000000003e-06 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660540+0000", 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4000000000000002e-06 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733304+0000", 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072764261999999996 2024-09-18T01:11:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733377+0000", 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2465999999999994e-05 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733623+0000", 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000246505 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225141 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 168960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.963019117, 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073470263999999993, 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225141 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660546+0000", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660552+0000", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0229999999999998e-06 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660558+0000", 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:11:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660563+0000", 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6580000000000001e-06 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733655+0000", 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073092007000000001 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.733739+0000", 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3755e-05 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734016+0000", 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027747399999999999 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225142 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 182272~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.962996117, 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074055758999999999, 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225142 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660569+0000", 2024-09-18T01:11:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5999999999999999e-07 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660571+0000", 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.979e-06 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660577+0000", 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660582+0000", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734046+0000", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073463801999999995 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734116+0000", 2024-09-18T01:11:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0426000000000006e-05 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734625+0000", 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050851300000000004 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225143 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 196608~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:46.660 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.96297744500001, 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074128059999999996, 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225143 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660588+0000", 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.96e-07 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660590+0000", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8369999999999999e-06 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660597+0000", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1840000000000002e-06 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660602+0000", 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6720000000000003e-06 2024-09-18T01:11:46.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734408+0000", 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073806415 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734479+0000", 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0986000000000001e-05 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734716+0000", 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023667 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225144 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 210944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.962957442, 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074961550000000002, 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225144 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660608+0000", 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4400000000000001e-07 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660610+0000", 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0470000000000001e-06 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660616+0000", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660621+0000", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734805+0000", 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074184342 2024-09-18T01:11:46.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.734877+0000", 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1607999999999996e-05 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735569+0000", 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069264800000000003 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225145 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.962939202, 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074764229000000001, 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225145 2024-09-18T01:11:46.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660626+0000", 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7300000000000002e-07 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660628+0000", 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8679999999999999e-06 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660634+0000", 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9340000000000003e-06 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660639+0000", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.685e-06 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735104+0000", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074464905999999997 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735173+0000", 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9196999999999995e-05 2024-09-18T01:11:46.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735390+0000", 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021736600000000001 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225146 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 238592~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.96292078800001, 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075011719000000004, 2024-09-18T01:11:46.674 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225146 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660644+0000", 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660645+0000", 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660647+0000", 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.103e-06 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660655+0000", 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660660+0000", 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0780000000000003e-06 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735409+0000", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074749776000000004 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735463+0000", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3130999999999997e-05 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735656+0000", 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019344499999999999 2024-09-18T01:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225147 3.b 3:d4e3551b:::rbd_data.1176c2733c33.0000000000000013:4a [read 252928~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.96289876100001, 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075344956000000005, 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225147 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660666+0000", 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660667+0000", 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660669+0000", 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4739999999999999e-06 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660677+0000", 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5680000000000002e-06 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.660682+0000", 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7770000000000002e-06 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735720+0000", 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.075038907000000002 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.735793+0000", 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2898999999999994e-05 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736011+0000", 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021788299999999999 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.683 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225325 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 32768~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958646982, 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072148217000000001, 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225325 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664918+0000", 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:46.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664920+0000", 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3599999999999999e-06 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664923+0000", 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664925+0000", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.689e-06 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736746+0000", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.071820175999999999 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.736817+0000", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1481000000000002e-05 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737066+0000", 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024918500000000002 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225326 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 47104~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.95863653000001, 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072466789000000004, 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225326 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664929+0000", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664930+0000", 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.288e-06 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664933+0000", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9739999999999998e-06 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664936+0000", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737095+0000", 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072159741999999999 2024-09-18T01:11:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737167+0000", 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1483999999999995e-05 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737395+0000", 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022858500000000001 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225327 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 61440~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958624839, 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072773537999999999, 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225327 2024-09-18T01:11:46.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:46.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664940+0000", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6899999999999999e-07 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664943+0000", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0930000000000001e-06 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664947+0000", 2024-09-18T01:11:46.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.904e-06 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664950+0000", 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.757e-06 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737425+0000", 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072474780000000003 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737504+0000", 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9264000000000005e-05 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737714+0000", 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000209471 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225328 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 74752~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.697 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958611542, 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073133644999999997, 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225328 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664954+0000", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664955+0000", 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.423e-06 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664958+0000", 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9050000000000001e-06 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664966+0000", 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737749+0000", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.072782989000000006 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.737862+0000", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112378 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738087+0000", 2024-09-18T01:11:46.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022564699999999999 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225329 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 89088~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958595471, 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073421049000000002, 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.702 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225329 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664970+0000", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664971+0000", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.672e-06 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664976+0000", 2024-09-18T01:11:46.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9570000000000001e-06 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664981+0000", 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.25e-06 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738122+0000", 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073141066000000005 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738183+0000", 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1342999999999999e-05 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738391+0000", 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000207531 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.706 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225330 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 103424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958578813, 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073692553999999993, 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225330 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664986+0000", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664987+0000", 2024-09-18T01:11:46.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664989+0000", 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664994+0000", 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.664999+0000", 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738416+0000", 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073416988000000002 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738472+0000", 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6198e-05 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738679+0000", 2024-09-18T01:11:46.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000206345 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225331 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 117760~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958559518, 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073976353999999994, 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225331 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:46.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665006+0000", 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665015+0000", 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2070000000000001e-06 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665022+0000", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6830000000000002e-06 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665027+0000", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738697+0000", 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073669942000000002 2024-09-18T01:11:46.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738775+0000", 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8455999999999996e-05 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.738982+0000", 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020658699999999999 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225332 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 131072~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958531288, 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074237658999999998, 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225332 2024-09-18T01:11:46.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665034+0000", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.27e-07 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665036+0000", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0530000000000001e-06 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665043+0000", 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9229999999999996e-06 2024-09-18T01:11:46.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665048+0000", 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7160000000000002e-06 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739011+0000", 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.073963685000000001 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739084+0000", 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2118000000000003e-05 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739271+0000", 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000187837 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4470.0:225333 3.b 3:d50006a6:::rbd_data.1176c2733c33.000000000000001e:4a [read 145408~5120 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-18T01:11:46.722 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 102.958510164, 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074510494999999996, 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4470", 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 225333 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-18T01:11:46.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665055+0000", 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.41e-07 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665057+0000", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1619999999999998e-06 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665063+0000", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5670000000000002e-06 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.665069+0000", 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8579999999999996e-06 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739302+0000", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.074233194000000002 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739362+0000", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9644e-05 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-18T01:10:03.739565+0000", 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020372899999999999 2024-09-18T01:11:46.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.727 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.727 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-18T01:11:46.727 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-18T01:11:46.727 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:46.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:46.729 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:46.727+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:46.829 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:46.828+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:46.851 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:46.851 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:46.851 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:46.851 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:46.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:46.946 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:46.945+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:47.003 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.003 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:47.003 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:47.004 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:47.004 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.012 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:47.031 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-18T01:11:47.030+0000 7fd77e495640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33331) UID: 0 2024-09-18T01:11:47.132 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:47.131+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.157 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:28511 3.a 3:53337fe2:::rbd_data.1176e641238f.000000000000000b:head [write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 229.43231542800001, 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92237777899999995, 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 28511 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722741+0000", 2024-09-18T01:11:47.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722750+0000", 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5650000000000002e-06 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722758+0000", 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8199999999999997e-06 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722762+0000", 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722817+0000", 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4069e-05 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:57.722823+0000", 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3500000000000002e-06 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640539+0000", 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.91771605099999998 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640677+0000", 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013767400000000001 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.640769+0000", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2649e-05 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.644881+0000", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041112739999999998 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645052+0000", 2024-09-18T01:11:47.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017164799999999999 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645089+0000", 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6400999999999999e-05 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:58.645118+0000", 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9836999999999999e-05 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186230 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 98304~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401288906, 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85753960699999998, 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186230 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753767+0000", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.28e-07 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753768+0000", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.071e-06 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753772+0000", 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.123e-06 2024-09-18T01:11:47.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753774+0000", 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2910000000000002e-06 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754739+0000", 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096504100000000003 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754746+0000", 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7780000000000002e-06 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611038+0000", 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85629203300000001 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611120+0000", 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1897999999999994e-05 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611307+0000", 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018714400000000001 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186231 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401279328, 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85784764400000002, 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.170 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186231 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753777+0000", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1e-07 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753778+0000", 2024-09-18T01:11:47.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.015e-06 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753782+0000", 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.676e-06 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753786+0000", 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754776+0000", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099006700000000007 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754782+0000", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5599999999999999e-06 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611339+0000", 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85655726099999996 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611419+0000", 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9238000000000003e-05 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611624+0000", 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020569699999999999 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186232 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401265302, 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85819400099999998, 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186232 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.176 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753791+0000", 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4700000000000002e-07 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753793+0000", 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753800+0000", 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753805+0000", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0350000000000001e-06 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754811+0000", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001006032 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754817+0000", 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-18T01:11:47.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611652+0000", 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85683446900000004 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611744+0000", 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1998999999999998e-05 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.611985+0000", 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024105599999999999 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186233 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 149504~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401236529, 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85849568600000004, 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.181 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186233 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753820+0000", 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753822+0000", 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0660000000000002e-06 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753830+0000", 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6869999999999994e-06 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753835+0000", 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3410000000000002e-06 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754948+0000", 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011132189999999999 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754956+0000", 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8909999999999995e-06 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612024+0000", 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85706775199999996 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612099+0000", 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5185000000000006e-05 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612315+0000", 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021610499999999999 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186234 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401208833, 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85882400800000003, 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.187 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186234 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753847+0000", 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753848+0000", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753850+0000", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2010000000000002e-06 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753857+0000", 2024-09-18T01:11:47.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1720000000000001e-06 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753862+0000", 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2079999999999999e-06 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755001+0000", 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001138745 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755009+0000", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7689999999999996e-06 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612346+0000", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85733753000000001 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612430+0000", 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3991000000000003e-05 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612671+0000", 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024085900000000001 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186235 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 194560~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401174385, 2024-09-18T01:11:47.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85916566999999999, 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186235 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753882+0000", 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753884+0000", 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.052e-06 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753893+0000", 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5890000000000003e-06 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753903+0000", 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604999999999999e-05 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755044+0000", 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001140612 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755050+0000", 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:11:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612720+0000", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85766986499999998 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.612797+0000", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7352000000000001e-05 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613047+0000", 2024-09-18T01:11:47.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025002800000000001 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186236 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 200704~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401144539, 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85940878799999998, 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186236 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:47.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753912+0000", 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753914+0000", 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753922+0000", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0350000000000001e-06 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753927+0000", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755212+0000", 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012846960000000001 2024-09-18T01:11:47.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755221+0000", 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3659999999999998e-06 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613077+0000", 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85785601199999995 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613146+0000", 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8664999999999994e-05 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613320+0000", 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017450000000000001 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186237 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:47.203 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401122348, 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85964439800000003, 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186237 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753934+0000", 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753936+0000", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.886e-06 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753942+0000", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4740000000000002e-06 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753949+0000", 2024-09-18T01:11:47.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1279999999999996e-06 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755255+0000", 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013061259999999999 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755262+0000", 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0119999999999999e-06 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613349+0000", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85808727299999998 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613417+0000", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7547999999999998e-05 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613578+0000", 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000161559 2024-09-18T01:11:47.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186238 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 212992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401085442, 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85988005000000001, 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186238 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:47.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753971+0000", 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.51e-07 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753973+0000", 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753980+0000", 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1539999999999996e-06 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753985+0000", 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755287+0000", 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013015990000000001 2024-09-18T01:11:47.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755293+0000", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613605+0000", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85831236499999997 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613677+0000", 2024-09-18T01:11:47.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1833000000000002e-05 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613851+0000", 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017341200000000001 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186239 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 251904~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401064327, 2024-09-18T01:11:47.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86015984700000003, 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186239 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:47.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753992+0000", 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.753994+0000", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754001+0000", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7220000000000002e-06 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754006+0000", 2024-09-18T01:11:47.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755327+0000", 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013216460000000001 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755334+0000", 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6680000000000004e-06 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613890+0000", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.858556243 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.613982+0000", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1570000000000006e-05 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614152+0000", 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000169863 2024-09-18T01:11:47.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186240 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 258048~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.401033802, 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.86041203399999999, 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186240 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754022+0000", 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754023+0000", 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754028+0000", 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6250000000000004e-06 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754037+0000", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4160000000000004e-06 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.754046+0000", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7360000000000007e-06 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755400+0000", 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001354673 2024-09-18T01:11:47.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.755408+0000", 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3459999999999996e-06 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614177+0000", 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85876956100000001 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614252+0000", 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5032999999999998e-05 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.614434+0000", 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018212 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186381 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 150528~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395553548, 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85787131900000002, 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186381 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759502+0000", 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759503+0000", 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7899999999999999e-07 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759505+0000", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759512+0000", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.516e-06 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759517+0000", 2024-09-18T01:11:47.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.994e-06 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000242768 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759767+0000", 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1049999999999995e-06 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617006+0000", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85723853699999997 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617082+0000", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5844000000000001e-05 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617374+0000", 2024-09-18T01:11:47.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029201699999999999 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186382 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395499689, 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85817453300000002, 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.231 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186382 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759556+0000", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759557+0000", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999996e-07 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759560+0000", 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9720000000000001e-06 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759568+0000", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1419999999999993e-06 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759575+0000", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759819+0000", 2024-09-18T01:11:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024391199999999999 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759827+0000", 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0129999999999993e-06 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617416+0000", 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85758962000000005 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617503+0000", 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6757999999999998e-05 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617731+0000", 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022796599999999999 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186383 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 195584~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395422267, 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85840259399999996, 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186383 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759634+0000", 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759637+0000", 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1030000000000002e-06 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759647+0000", 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.781e-06 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759654+0000", 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9800000000000001e-06 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759859+0000", 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020511600000000001 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759867+0000", 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.401e-06 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617761+0000", 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85789394799999996 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.617815+0000", 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4070999999999998e-05 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618036+0000", 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000221623 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186384 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 201728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395392153, 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859935899999995, 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186384 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759664+0000", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759667+0000", 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-18T01:11:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759674+0000", 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0740000000000004e-06 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759679+0000", 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2440000000000001e-06 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759902+0000", 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000222952 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759909+0000", 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.204e-06 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618071+0000", 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85816230699999996 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618124+0000", 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2302000000000002e-05 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618263+0000", 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013966799999999999 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186385 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 207872~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395342235, 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85884324199999995, 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186385 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759714+0000", 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759715+0000", 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1900000000000001e-07 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759718+0000", 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7419999999999999e-06 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759729+0000", 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-05 2024-09-18T01:11:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759737+0000", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4259999999999997e-06 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759936+0000", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019923099999999999 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759943+0000", 2024-09-18T01:11:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6189999999999996e-06 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618302+0000", 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85835968100000004 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618384+0000", 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1298999999999995e-05 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618557+0000", 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173435 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186386 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 214016~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395308648, 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85900794700000005, 2024-09-18T01:11:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186386 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759747+0000", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759748+0000", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8500000000000002e-07 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759751+0000", 2024-09-18T01:11:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4290000000000001e-06 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759759+0000", 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759764+0000", 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.592e-06 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759975+0000", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021040300000000001 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759982+0000", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1319999999999996e-06 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618575+0000", 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85859339199999996 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618627+0000", 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1657999999999999e-05 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618755+0000", 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012844000000000001 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186387 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 252928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.39528411400001, 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85925986600000004, 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186387 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759772+0000", 2024-09-18T01:11:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3500000000000002e-07 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759774+0000", 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.097e-06 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759781+0000", 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7950000000000002e-06 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759786+0000", 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9200000000000003e-06 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760010+0000", 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022365899999999999 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760015+0000", 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3850000000000001e-06 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618792+0000", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85877653200000004 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.618868+0000", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6346000000000004e-05 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619032+0000", 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163697 2024-09-18T01:11:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186388 3.9 3:9275b10c:::rbd_data.11762ddd5718.000000000000001b:48 [read 259072~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.395098832, 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85926423299999999, 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186388 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759957+0000", 2024-09-18T01:11:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759958+0000", 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2500000000000005e-07 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759962+0000", 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759971+0000", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5730000000000008e-06 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.759977+0000", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.404e-06 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760069+0000", 2024-09-18T01:11:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2385000000000004e-05 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:56.760075+0000", 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619050+0000", 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.85897480100000001 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619092+0000", 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2109e-05 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.619221+0000", 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012919400000000001 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.269 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.271 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:47.271 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:47.270+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:47.372 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:47.371+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:47.415 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.415 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:47.415 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:47.415 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:47.473 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:47.472+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:47.528 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.528 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:47.529 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:47.529 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:47.529 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:47.573 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:47.572+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:47.674 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-18T01:11:47.673+0000 7f677ecd8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-18T01:11:47.683 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.683 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:47.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:47.683 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:47.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188295 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 49152~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08221421499999, 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043458713000000003, 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188295 2024-09-18T01:11:47.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599518+0000", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599520+0000", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.877e-06 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599526+0000", 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4829999999999999e-06 2024-09-18T01:11:47.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.599531+0000", 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.288e-06 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642834+0000", 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043302606 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642891+0000", 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7973999999999998e-05 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.642977+0000", 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5152000000000001e-05 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188327 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 55296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.0810642, 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044734847000000001, 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188327 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600668+0000", 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600670+0000", 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.849e-06 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600676+0000", 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8220000000000003e-06 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600681+0000", 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7500000000000003e-06 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643614+0000", 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042933702999999997 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.643688+0000", 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3924999999999994e-05 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645403+0000", 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001714442 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188340 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.081044889, 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044908370000000003, 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188340 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600687+0000", 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600688+0000", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.75e-07 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600696+0000", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4849999999999992e-06 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600704+0000", 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.238e-06 2024-09-18T01:11:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600709+0000", 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645436+0000", 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044726941999999999 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645508+0000", 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1993e-05 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645596+0000", 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8078000000000006e-05 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188341 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.081017143, 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081629999999998, 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188341 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600715+0000", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600718+0000", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1169999999999998e-06 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600724+0000", 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4189999999999999e-06 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600729+0000", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5310000000000003e-06 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645624+0000", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044895178000000001 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645688+0000", 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3938999999999999e-05 2024-09-18T01:11:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645797+0000", 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010901 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188342 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080997291, 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045237065, 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188342 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600735+0000", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.27e-07 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600737+0000", 2024-09-18T01:11:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0379999999999998e-06 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600743+0000", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999998e-06 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600748+0000", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6210000000000003e-06 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645829+0000", 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045081230999999999 2024-09-18T01:11:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645893+0000", 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3458000000000001e-05 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.645972+0000", 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9250000000000002e-05 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188343 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08097742699999, 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045462521999999998, 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188343 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600755+0000", 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.34e-07 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600757+0000", 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:11:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600763+0000", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7679999999999997e-06 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600768+0000", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646023+0000", 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045255086 2024-09-18T01:11:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646113+0000", 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0266999999999995e-05 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646217+0000", 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104256 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188344 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 89088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080959221, 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045696480999999997, 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188344 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600773+0000", 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2800000000000003e-07 2024-09-18T01:11:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600775+0000", 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.018e-06 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600782+0000", 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600787+0000", 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646269+0000", 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045482013000000002 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646369+0000", 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9726e-05 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646469+0000", 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100177 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188345 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08093917399999, 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046527988999999999, 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188345 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600793+0000", 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.89e-07 2024-09-18T01:11:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600795+0000", 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600801+0000", 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.823e-06 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600806+0000", 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6269999999999999e-06 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646501+0000", 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.045695278999999998 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.646576+0000", 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4618999999999998e-05 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647321+0000", 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00074519099999999997 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188346 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 101376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08092096599999, 2024-09-18T01:11:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046723897, 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188346 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600811+0000", 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600812+0000", 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600813+0000", 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.77e-06 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600819+0000", 2024-09-18T01:11:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:11:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600824+0000", 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6789999999999996e-06 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647347+0000", 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046522844000000001 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647436+0000", 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9023999999999993e-05 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647535+0000", 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9449000000000003e-05 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188347 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 107520~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080902854, 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047172448999999998, 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188347 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600829+0000", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600830+0000", 2024-09-18T01:11:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600831+0000", 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.731e-06 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600837+0000", 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7570000000000001e-06 2024-09-18T01:11:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600842+0000", 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7380000000000002e-06 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647562+0000", 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.046719787999999998 2024-09-18T01:11:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.647645+0000", 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2949999999999997e-05 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648002+0000", 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035715500000000001 2024-09-18T01:11:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188348 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080884936, 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047548153000000003, 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188348 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600847+0000", 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600848+0000", 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3999999999999997e-07 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600849+0000", 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600855+0000", 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7950000000000004e-06 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600860+0000", 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8040000000000001e-06 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648030+0000", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047169606000000003 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648100+0000", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0902000000000007e-05 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648395+0000", 2024-09-18T01:11:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029488700000000002 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188349 3.16 3:6d29a324:::rbd_data.11762ddd5718.000000000000002e:48 [read 152576~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08086691600001, 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048288559000000002, 2024-09-18T01:11:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188349 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600865+0000", 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600866+0000", 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4700000000000001e-07 2024-09-18T01:11:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600873+0000", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.126e-06 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600879+0000", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.600884+0000", 2024-09-18T01:11:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648421+0000", 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047537215000000001 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.648564+0000", 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014283699999999999 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649154+0000", 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058960900000000005 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188426 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 31744~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08031396, 2024-09-18T01:11:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047891319000000002, 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188426 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601418+0000", 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601419+0000", 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.75e-07 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601421+0000", 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:11:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601427+0000", 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8459999999999996e-06 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601431+0000", 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6700000000000002e-06 2024-09-18T01:11:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649198+0000", 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047766690000000001 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649253+0000", 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4441000000000003e-05 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649310+0000", 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6925999999999997e-05 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188427 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 37888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080295237, 2024-09-18T01:11:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048301442999999999, 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188427 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601437+0000", 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.34e-07 2024-09-18T01:11:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601439+0000", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601445+0000", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.314e-06 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601460+0000", 2024-09-18T01:11:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4398e-05 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649430+0000", 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.047970278999999998 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649506+0000", 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5456000000000004e-05 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649738+0000", 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023279900000000001 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188428 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 44032~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080265454, 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048459213000000001, 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188428 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601467+0000", 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601476+0000", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5569999999999996e-06 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601482+0000", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601488+0000", 2024-09-18T01:11:47.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.152e-06 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649766+0000", 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048278190999999998 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649834+0000", 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8172999999999999e-05 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649926+0000", 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2e-05 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188429 3.16 3:690b6a70:::rbd_data.11762ddd5718.0000000000000036:48 [read 50176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.080201366, 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048567179000000002, 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188429 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601531+0000", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3900000000000005e-07 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601534+0000", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5150000000000001e-06 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601540+0000", 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.263e-06 2024-09-18T01:11:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601545+0000", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.649952+0000", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048406656999999999 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650021+0000", 2024-09-18T01:11:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9128000000000006e-05 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650098+0000", 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6746e-05 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188458 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 5120~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.08001706500001, 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048579575999999999, 2024-09-18T01:11:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188458 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601715+0000", 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601717+0000", 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9089999999999998e-06 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601723+0000", 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6930000000000004e-06 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601728+0000", 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5519999999999998e-06 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650142+0000", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048414361000000003 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650228+0000", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5765000000000003e-05 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650295+0000", 2024-09-18T01:11:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6933000000000004e-05 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188459 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 11264~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.079998911, 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048722040000000001, 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188459 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601733+0000", 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601734+0000", 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.27e-07 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601744+0000", 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0402e-05 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601750+0000", 2024-09-18T01:11:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1800000000000001e-06 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601755+0000", 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8579999999999999e-06 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650327+0000", 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048572024999999998 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650373+0000", 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6371e-05 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650455+0000", 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188460 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 17408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.079970539, 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048965328000000002, 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188460 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601762+0000", 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601764+0000", 2024-09-18T01:11:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1160000000000002e-06 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601771+0000", 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9369999999999998e-06 2024-09-18T01:11:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601776+0000", 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1070000000000004e-06 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650480+0000", 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.048703966000000001 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650539+0000", 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9302000000000002e-05 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650727+0000", 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018754700000000001 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:188461 3.16 3:6cb65339:::rbd_data.11762ddd5718.000000000000003e:48 [read 56320~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.079948613, 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049366139000000003, 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 188461 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601784+0000", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0999999999999999e-07 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601786+0000", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9690000000000001e-06 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601792+0000", 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4230000000000002e-06 2024-09-18T01:11:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.601797+0000", 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659999999999997e-06 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650805+0000", 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049008068000000002 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.650913+0000", 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107941 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.651150+0000", 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236462 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:47.790 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.790 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:11:47.793 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:47.793 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:47.791+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:47.893 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:47.892+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:47.937 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:47.937 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:47.938 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:11:47.938 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:47.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:47.994 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-18T01:11:47.993+0000 7fa079d48640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33237) UID: 0 2024-09-18T01:11:48.051 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:11:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-18T01:11:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-18T01:11:48.051 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:48.059 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:48.100 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:48.099+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:48.196 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:48.195+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4234.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 276.305161833, 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096447879999999, 2024-09-18T01:11:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4234", 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/858475858", 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897809+0000", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897811+0000", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897818+0000", 2024-09-18T01:11:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5159999999999997e-06 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897831+0000", 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3037000000000001e-05 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897842+0000", 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1151999999999999e-05 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897937+0000", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4401999999999995e-05 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:11.897953+0000", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6181e-05 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906149+0000", 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0081959300000001 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906311+0000", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016237900000000001 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.906469+0000", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015766700000000001 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907083+0000", 2024-09-18T01:11:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00061415999999999999 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907349+0000", 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026650300000000002 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907409+0000", 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9778000000000003e-05 2024-09-18T01:11:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:07:12.907453+0000", 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4233000000000002e-05 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:65906 3.15 3:adfc2d2e:::rbd_data.117658cbc921.0000000000000001:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 174.581211282, 2024-09-18T01:11:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057681414, 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 65906 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621759+0000", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621760+0000", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621762+0000", 2024-09-18T01:11:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0209999999999998e-06 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621769+0000", 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.621775+0000", 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679286+0000", 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.057510998000000001 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679359+0000", 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3796999999999998e-05 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:08:53.679441+0000", 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1259000000000002e-05 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:123373 3.e 3:702911f8:::rbd_data.1176e38d32d8.0000000000000002:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 138.613706815, 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97448278200000005, 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 123373 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589264+0000", 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8899999999999999e-07 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589268+0000", 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5640000000000001e-06 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589275+0000", 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.815e-06 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589280+0000", 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589339+0000", 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8353999999999997e-05 2024-09-18T01:11:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:29.589351+0000", 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2248e-05 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561340+0000", 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.971989714 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561473+0000", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013203499999999999 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.561648+0000", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017573400000000001 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563011+0000", 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013627019999999999 2024-09-18T01:11:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563658+0000", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064681700000000005 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563704+0000", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6316000000000002e-05 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:30.563746+0000", 2024-09-18T01:11:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2401999999999998e-05 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186541 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.608734486, 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98458378700000004, 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186541 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594236+0000", 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594237+0000", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594241+0000", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594253+0000", 2024-09-18T01:11:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1484000000000001e-05 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594260+0000", 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.187e-06 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594343+0000", 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3047999999999995e-05 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.594355+0000", 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1861999999999999e-05 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578696+0000", 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.984341511 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.578820+0000", 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012341699999999999 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186561 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.60384210300001, 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.98006722099999999, 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186561 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599128+0000", 2024-09-18T01:11:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599129+0000", 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599133+0000", 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0389999999999998e-06 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599142+0000", 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7700000000000007e-06 2024-09-18T01:11:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599148+0000", 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2389999999999999e-06 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599223+0000", 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4371999999999999e-05 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.599237+0000", 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4382000000000001e-05 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579132+0000", 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97989460299999998 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579196+0000", 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.41e-05 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186581 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.598427465, 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97473331600000002, 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186581 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604543+0000", 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604544+0000", 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2200000000000003e-07 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604548+0000", 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:11:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604558+0000", 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0711e-05 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604565+0000", 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0990000000000001e-06 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604627+0000", 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1673000000000004e-05 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.604634+0000", 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7059999999999998e-06 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579227+0000", 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.97459336200000002 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579276+0000", 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9208e-05 2024-09-18T01:11:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186898 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.586853199, 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96323869900000003, 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186898 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616117+0000", 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616118+0000", 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616121+0000", 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6730000000000001e-06 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616131+0000", 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0471e-05 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616137+0000", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7749999999999998e-06 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616226+0000", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9277999999999995e-05 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.616235+0000", 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2930000000000006e-06 2024-09-18T01:11:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579302+0000", 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96306649799999999 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579356+0000", 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4103e-05 2024-09-18T01:11:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186909 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.577885506, 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95434538800000002, 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186909 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625085+0000", 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625087+0000", 2024-09-18T01:11:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.762e-06 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625093+0000", 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3829999999999997e-06 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625110+0000", 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6464999999999999e-05 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625118+0000", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3080000000000008e-06 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625209+0000", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1281999999999998e-05 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625221+0000", 2024-09-18T01:11:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579385+0000", 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95416372599999999 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579430+0000", 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5646999999999999e-05 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186914 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.57783104400001, 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95469037999999995, 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186914 2024-09-18T01:11:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625139+0000", 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625140+0000", 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625143+0000", 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625151+0000", 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625156+0000", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625283+0000", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012635599999999999 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.625292+0000", 2024-09-18T01:11:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2220000000000008e-06 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579546+0000", 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95425370499999995 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579686+0000", 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140817 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579830+0000", 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014347900000000001 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186941 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.57255967, 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94910140600000004, 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186941 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630411+0000", 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630412+0000", 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6080000000000001e-06 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630420+0000", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5399999999999998e-06 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630437+0000", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7306e-05 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630458+0000", 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0389000000000001e-05 2024-09-18T01:11:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630598+0000", 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014037699999999999 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630607+0000", 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5339999999999999e-06 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579460+0000", 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94885344500000002 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579512+0000", 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2206999999999997e-05 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186946 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.57247893100001, 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94990149800000001, 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186946 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630492+0000", 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630496+0000", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9570000000000004e-06 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630507+0000", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0832999999999999e-05 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630514+0000", 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6969999999999996e-06 2024-09-18T01:11:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630563+0000", 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8999999999999998e-05 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.630573+0000", 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0562e-05 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580122+0000", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.949548422 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580210+0000", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8315999999999999e-05 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580393+0000", 2024-09-18T01:11:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018305099999999999 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186968 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.57151793200001, 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94813059300000002, 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186968 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631453+0000", 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631458+0000", 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9099999999999996e-06 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631473+0000", 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4854e-05 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631481+0000", 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1089999999999998e-06 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631553+0000", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1806999999999999e-05 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631565+0000", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2160999999999999e-05 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579536+0000", 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94797107899999999 2024-09-18T01:11:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579583+0000", 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6826000000000002e-05 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186973 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.57145914, 2024-09-18T01:11:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94905934199999997, 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186973 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631511+0000", 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631512+0000", 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631516+0000", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.529e-06 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631525+0000", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1309999999999995e-06 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631531+0000", 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631582+0000", 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0742999999999999e-05 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.631592+0000", 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0053e-05 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580423+0000", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94883132299999995 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580491+0000", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8025e-05 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580571+0000", 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9808000000000006e-05 2024-09-18T01:11:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:186995 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.570295172, 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94698495800000004, 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 186995 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632675+0000", 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632676+0000", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3799999999999996e-07 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632681+0000", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5190000000000003e-06 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632694+0000", 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3597999999999999e-05 2024-09-18T01:11:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632703+0000", 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000007e-06 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632766+0000", 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3454000000000006e-05 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.632778+0000", 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1058e-05 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579614+0000", 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94683610399999996 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579660+0000", 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6687000000000002e-05 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187000 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.56993050299999, 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94773635199999995, 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187000 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633040+0000", 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633041+0000", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2099999999999995e-07 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633046+0000", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7670000000000003e-06 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633059+0000", 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2962000000000001e-05 2024-09-18T01:11:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633066+0000", 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3919999999999997e-06 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633121+0000", 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4874999999999999e-05 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633131+0000", 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8570000000000007e-06 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580618+0000", 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94748755500000004 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580690+0000", 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000006e-05 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580776+0000", 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6522999999999998e-05 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187023 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.569407967, 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94616537599999995, 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187023 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633563+0000", 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633568+0000", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6349999999999997e-06 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633582+0000", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633590+0000", 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0590000000000003e-06 2024-09-18T01:11:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633737+0000", 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000147386 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633747+0000", 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9219999999999992e-06 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579684+0000", 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94593659200000002 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579728+0000", 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4298999999999997e-05 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187028 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.569010586, 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94700277600000005, 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187028 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633960+0000", 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633961+0000", 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633964+0000", 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7519999999999998e-06 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633973+0000", 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8990000000000007e-06 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.633978+0000", 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4150000000000001e-06 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634026+0000", 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8649000000000001e-05 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.634036+0000", 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2569999999999996e-06 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580802+0000", 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94676686300000001 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580878+0000", 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5679999999999994e-05 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580963+0000", 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4542999999999993e-05 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187050 3.d 3:b30b5a76:::rbd_data.11762ddd5718.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.56715785599999, 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.943974439, 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187050 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635813+0000", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1699999999999997e-07 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635818+0000", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635827+0000", 2024-09-18T01:11:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5799999999999998e-06 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635834+0000", 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.934e-06 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635896+0000", 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1235000000000006e-05 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635914+0000", 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8563000000000002e-05 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579743+0000", 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94382922999999996 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.579787+0000", 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3804999999999998e-05 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:187055 3.e 3:75676d3e:::rbd_data.11762ddd5718.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.56703566500001, 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94520640700000003, 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 187055 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635935+0000", 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635936+0000", 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1800000000000005e-07 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635944+0000", 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0560000000000005e-06 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635952+0000", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7129999999999994e-06 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.635959+0000", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5050000000000004e-06 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636014+0000", 2024-09-18T01:11:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4920999999999998e-05 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:57.636023+0000", 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4919999999999999e-06 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.580989+0000", 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94496594499999997 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581057+0000", 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8016999999999996e-05 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:09:58.581141+0000", 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4040000000000002e-05 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4470.0:258055 3.19 3:9ff6078a:::rbd_id.image_client.0-clone23:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e249)", 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.563544987, 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.95186631499999996, 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "continuous": false, 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4470", 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.120:0/2398965112", 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 258055 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639426+0000", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639427+0000", 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9410000000000001e-06 2024-09-18T01:11:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639434+0000", 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8730000000000001e-06 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639451+0000", 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6594000000000001e-05 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639458+0000", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4789999999999999e-06 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639509+0000", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0645e-05 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:24.639516+0000", 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1810000000000004e-06 2024-09-18T01:11:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588228+0000", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.94871193499999995 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588377+0000", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014855700000000001 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.588500+0000", 2024-09-18T01:11:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000123497 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.590468+0000", 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019678320000000001 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591148+0000", 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068034900000000001 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-18T01:11:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591217+0000", 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8204e-05 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-18T01:11:25.591292+0000", 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5228000000000005e-05 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-18T01:11:48.324 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:11:48.326 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-18T01:11:48.325+0000 7f803f7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33238) UID: 0 2024-09-18T01:11:48.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:11:48.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:11:48.426 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-18T01:11:48.425+0000 7f45a5201640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-18T01:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-18T01:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-18T01:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:11:48.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:11:48.449 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-18T01:11:48.449 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:11:48.467 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-18T01:11:48.467 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:11:48.483 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-18T01:11:48.483 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:11:48.499 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-18T01:11:48.500 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:11:48.516 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-18T01:11:48.517 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:11:48.527 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:48.526+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:48.533 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-18T01:11:48.534 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:11:48.534 INFO:teuthology.orchestra.run.smithi120.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-18T01:11:48.550 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-18T01:11:48.550 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:11:48.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T01:11:48.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T01:11:48.554 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:48.554 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:48.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:11:48.648 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-18T01:11:48.647+0000 7fae028d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33339) UID: 0 2024-09-18T01:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:48.709 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:48.709 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:48.722 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:11:48.728 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-09-18T01:11:48.727+0000 7f5417e80640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33245) UID: 0 2024-09-18T01:11:48.812 INFO:teuthology.orchestra.run.smithi120.stdout:All operations completed A-OK! 2024-09-18T01:11:48.818 DEBUG:teuthology.parallel:result is None 2024-09-18T01:11:48.819 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-18T01:11:48.830 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T01:11:48.889 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T01:11:48.890 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:48.891 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:48.891 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:11:48.993 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:48.993 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.002 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:11:49.105 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.106 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:11:49.217 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.217 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:11:49.328 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.328 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.337 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:11:49.439 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.439 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:11:49.552 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.552 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.560 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:11:49.662 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.663 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:11:49.775 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.775 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.784 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:11:49.894 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:49.895 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:49.903 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:11:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.006 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.015 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:11:50.119 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.119 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:11:50.232 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.233 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:11:50.345 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.346 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.354 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:11:50.458 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.459 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:11:50.570 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.571 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:11:50.683 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.683 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:50.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:11:50.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:11:50.797 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-18T01:11:50.797 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:11:51.024 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-18T01:11:51.024 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-18T01:11:51.024 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T01:11:51.024 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T01:11:51.027 INFO:tasks.ceph.osd.6:Started 2024-09-18T01:11:51.027 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:0 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:1 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:2 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:abort 2024-09-18T01:11:51.126 INFO:teuthology.orchestra.run.smithi162.stderr:assert 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:config diff 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:config diff get 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:config get 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:config help [] 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:config set ... 2024-09-18T01:11:51.127 INFO:teuthology.orchestra.run.smithi162.stderr:admin_socket: invalid command 2024-09-18T01:11:51.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:51.129 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-18T01:11:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-18T01:11:51.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:11:52.682 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:11:52.680+0000 7f44d600d640 -1 Falling back to public interface 2024-09-18T01:11:52.839 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-18T01:11:56.130 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:no valid command found; 10 closest matches: 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:0 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:1 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:2 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:abort 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:assert 2024-09-18T01:11:56.246 INFO:teuthology.orchestra.run.smithi162.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:11:56.247 INFO:teuthology.orchestra.run.smithi162.stderr:bluefs files list 2024-09-18T01:11:56.247 INFO:teuthology.orchestra.run.smithi162.stderr:bluefs stats 2024-09-18T01:11:56.247 INFO:teuthology.orchestra.run.smithi162.stderr:bluestore allocator dump block 2024-09-18T01:11:56.247 INFO:teuthology.orchestra.run.smithi162.stderr:bluestore allocator fragmentation block 2024-09-18T01:11:56.247 INFO:teuthology.orchestra.run.smithi162.stderr:admin_socket: invalid command 2024-09-18T01:11:56.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:11:56.249 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-18T01:11:57.197 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:11:57.196+0000 7f44d600d640 -1 osd.6 217 log_to_monitors true 2024-09-18T01:11:57.900 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-09-18T01:11:57.899+0000 7f44c970d640 -1 osd.6 276 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:12:01.250 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:12:01.355 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-18T01:12:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-18T01:12:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-18T01:12:01.355 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-18T01:12:01.364 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T01:12:01.487 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a: {} 2024-09-18T01:12:01.488 INFO:teuthology.orchestra.run.smithi120.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:12:01.497 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b: {} 2024-09-18T01:12:01.497 INFO:teuthology.orchestra.run.smithi120.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:12:01.505 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c: {} 2024-09-18T01:12:01.505 INFO:teuthology.orchestra.run.smithi120.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:12:01.516 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:12:02.590 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:12:02.611 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T01:12:03.599 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T01:12:03.618 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:12:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:12:04.618 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T01:12:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T01:12:05.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:12:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:12:06.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T01:12:07.628 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T01:12:07.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:12:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:12:08.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T01:12:09.630 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T01:12:09.648 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:12:10.642 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:12:10.659 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T01:12:11.646 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T01:12:11.662 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:12:12.652 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:12:12.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T01:12:13.666 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T01:12:13.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:12:14.586 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:12:14.602 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T01:12:15.673 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T01:12:15.690 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:12:16.675 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:12:16.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T01:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T01:12:17.702 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T01:12:17.702 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T01:12:17.702 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T01:12:17.702 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T01:12:17.702 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T01:12:17.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:12:17.983 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:17.984 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":296,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:12:17.679602+0000","last_up_change":"2024-09-18T01:11:58.860615+0000","last_in_change":"2024-09-18T01:10:43.698009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:07:17.736243+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:07:27.050688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"273","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":273,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":277,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":217,"up_from":277,"up_thru":281,"down_at":219,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6809","nonce":1013313713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6811","nonce":1013313713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6815","nonce":1013313713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6813","nonce":1013313713}]},"public_addr":"172.21.15.162:6809/1013313713","cluster_addr":"172.21.15.162:6811/1013313713","heartbeat_back_addr":"172.21.15.162:6815/1013313713","heartbeat_front_addr":"172.21.15.162:6813/1013313713","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":219},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7},{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:12:17.996 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T01:12:17.997 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:12:18.275 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:18.275 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":296,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:12:17.679602+0000","last_up_change":"2024-09-18T01:11:58.860615+0000","last_in_change":"2024-09-18T01:10:43.698009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:07:17.736243+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:07:27.050688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"273","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":273,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":277,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":217,"up_from":277,"up_thru":281,"down_at":219,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6809","nonce":1013313713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6811","nonce":1013313713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6815","nonce":1013313713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6813","nonce":1013313713}]},"public_addr":"172.21.15.162:6809/1013313713","cluster_addr":"172.21.15.162:6811/1013313713","heartbeat_back_addr":"172.21.15.162:6815/1013313713","heartbeat_front_addr":"172.21.15.162:6813/1013313713","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":219},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7},{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:12:18.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T01:12:18.289 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T01:12:18.289 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T01:12:18.289 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T01:12:18.290 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T01:12:18.290 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T01:12:18.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T01:12:18.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T01:12:18.520 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:18.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:12:18.531 INFO:teuthology.orchestra.run.smithi120.stdout:64424509505 2024-09-18T01:12:18.531 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:12:18.533 INFO:teuthology.orchestra.run.smithi120.stdout:55834574914 2024-09-18T01:12:18.533 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:12:18.534 INFO:teuthology.orchestra.run.smithi120.stdout:1189705940998 2024-09-18T01:12:18.534 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:12:18.535 INFO:teuthology.orchestra.run.smithi120.stdout:55834574914 2024-09-18T01:12:18.536 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:12:18.536 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:18.537 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:12:18.538 INFO:teuthology.orchestra.run.smithi120.stdout:64424509508 2024-09-18T01:12:18.539 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:12:18.540 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:18.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:12:19.092 INFO:teuthology.orchestra.run.smithi120.stdout:55834574913 2024-09-18T01:12:19.099 INFO:teuthology.orchestra.run.smithi120.stdout:64424509503 2024-09-18T01:12:19.102 INFO:teuthology.orchestra.run.smithi120.stdout:64424509504 2024-09-18T01:12:19.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574914 got 55834574913 for osd.4 2024-09-18T01:12:19.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509505 got 64424509503 for osd.1 2024-09-18T01:12:19.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509504 for osd.2 2024-09-18T01:12:19.134 INFO:teuthology.orchestra.run.smithi120.stdout:64424509504 2024-09-18T01:12:19.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509504 for osd.0 2024-09-18T01:12:19.161 INFO:teuthology.orchestra.run.smithi120.stdout:1189705940996 2024-09-18T01:12:19.165 INFO:teuthology.orchestra.run.smithi120.stdout:55834574913 2024-09-18T01:12:19.172 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:19.177 INFO:teuthology.orchestra.run.smithi120.stdout:64424509504 2024-09-18T01:12:19.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 1189705940998 got 1189705940996 for osd.6 2024-09-18T01:12:19.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574914 got 55834574913 for osd.3 2024-09-18T01:12:19.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509508 got 64424509506 for osd.7 2024-09-18T01:12:19.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509504 for osd.5 2024-09-18T01:12:20.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:12:20.120 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:12:20.122 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:12:20.148 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:12:20.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:12:20.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:12:20.188 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:12:20.191 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:12:20.654 INFO:teuthology.orchestra.run.smithi120.stdout:64424509505 2024-09-18T01:12:20.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509505 got 64424509505 for osd.1 2024-09-18T01:12:20.674 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.677 INFO:teuthology.orchestra.run.smithi120.stdout:55834574914 2024-09-18T01:12:20.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574914 got 55834574914 for osd.4 2024-09-18T01:12:20.696 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.715 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:20.720 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:20.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509506 for osd.2 2024-09-18T01:12:20.731 INFO:teuthology.orchestra.run.smithi120.stdout:64424509506 2024-09-18T01:12:20.731 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509506 for osd.0 2024-09-18T01:12:20.740 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509506 got 64424509506 for osd.5 2024-09-18T01:12:20.744 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.764 INFO:teuthology.orchestra.run.smithi120.stdout:55834574914 2024-09-18T01:12:20.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574914 got 55834574914 for osd.3 2024-09-18T01:12:20.777 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.794 INFO:teuthology.orchestra.run.smithi120.stdout:1189705940998 2024-09-18T01:12:20.803 INFO:teuthology.orchestra.run.smithi120.stdout:64424509508 2024-09-18T01:12:20.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 1189705940998 got 1189705940998 for osd.6 2024-09-18T01:12:20.808 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509508 got 64424509508 for osd.7 2024-09-18T01:12:20.816 DEBUG:teuthology.parallel:result is None 2024-09-18T01:12:20.816 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:12:20.816 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:21.131 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:21.132 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:21.143 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-18T01:12:20.779048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517832,"kb_used_data":29196,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456888,"statfs":{"total":773094113280,"available":772563853312,"internally_reserved":0,"allocated":29896704,"data_stored":15583005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001714"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4802,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346221+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346221+0000","last_peered":"2024-09-18T01:12:18.346221+0000","last_clean":"2024-09-18T01:12:18.346221+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346221+0000","last_undegraded":"2024-09-18T01:12:18.346221+0000","last_fullsized":"2024-09-18T01:12:18.346221+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13518,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345866+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:18.345866+0000","last_peered":"2024-09-18T01:12:18.345866+0000","last_clean":"2024-09-18T01:12:18.345866+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:18.345866+0000","last_undegraded":"2024-09-18T01:12:18.345866+0000","last_fullsized":"2024-09-18T01:12:18.345866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:35.072079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794718+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794718+0000","last_peered":"2024-09-18T01:12:17.794718+0000","last_clean":"2024-09-18T01:12:17.794718+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794718+0000","last_undegraded":"2024-09-18T01:12:17.794718+0000","last_fullsized":"2024-09-18T01:12:17.794718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4756,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345763+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345763+0000","last_peered":"2024-09-18T01:12:18.345763+0000","last_clean":"2024-09-18T01:12:18.345763+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345763+0000","last_undegraded":"2024-09-18T01:12:18.345763+0000","last_fullsized":"2024-09-18T01:12:18.345763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10101,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346067+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346067+0000","last_peered":"2024-09-18T01:12:18.346067+0000","last_clean":"2024-09-18T01:12:18.346067+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346067+0000","last_undegraded":"2024-09-18T01:12:18.346067+0000","last_fullsized":"2024-09-18T01:12:18.346067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7832,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345666+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345666+0000","last_peered":"2024-09-18T01:12:18.345666+0000","last_clean":"2024-09-18T01:12:18.345666+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345666+0000","last_undegraded":"2024-09-18T01:12:18.345666+0000","last_fullsized":"2024-09-18T01:12:18.345666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:45.894849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8589,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346101+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346101+0000","last_peered":"2024-09-18T01:12:18.346101+0000","last_clean":"2024-09-18T01:12:18.346101+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346101+0000","last_undegraded":"2024-09-18T01:12:18.346101+0000","last_fullsized":"2024-09-18T01:12:18.346101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4928,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345917+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345917+0000","last_peered":"2024-09-18T01:12:18.345917+0000","last_clean":"2024-09-18T01:12:18.345917+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345917+0000","last_undegraded":"2024-09-18T01:12:18.345917+0000","last_fullsized":"2024-09-18T01:12:18.345917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:34.188190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5424,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345409+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345409+0000","last_peered":"2024-09-18T01:12:18.345409+0000","last_clean":"2024-09-18T01:12:18.345409+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345409+0000","last_undegraded":"2024-09-18T01:12:18.345409+0000","last_fullsized":"2024-09-18T01:12:18.345409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:07.663503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71360,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300480,"statfs":{"total":96636764160,"available":96563691520,"internally_reserved":0,"allocated":4169728,"data_stored":2385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:21.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:21.420 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:21.420 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:21.431 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-18T01:12:20.779048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517832,"kb_used_data":29196,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456888,"statfs":{"total":773094113280,"available":772563853312,"internally_reserved":0,"allocated":29896704,"data_stored":15583005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001714"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4802,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346221+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346221+0000","last_peered":"2024-09-18T01:12:18.346221+0000","last_clean":"2024-09-18T01:12:18.346221+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346221+0000","last_undegraded":"2024-09-18T01:12:18.346221+0000","last_fullsized":"2024-09-18T01:12:18.346221+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13518,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345866+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:18.345866+0000","last_peered":"2024-09-18T01:12:18.345866+0000","last_clean":"2024-09-18T01:12:18.345866+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:18.345866+0000","last_undegraded":"2024-09-18T01:12:18.345866+0000","last_fullsized":"2024-09-18T01:12:18.345866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:35.072079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794718+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794718+0000","last_peered":"2024-09-18T01:12:17.794718+0000","last_clean":"2024-09-18T01:12:17.794718+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794718+0000","last_undegraded":"2024-09-18T01:12:17.794718+0000","last_fullsized":"2024-09-18T01:12:17.794718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4756,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345763+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345763+0000","last_peered":"2024-09-18T01:12:18.345763+0000","last_clean":"2024-09-18T01:12:18.345763+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345763+0000","last_undegraded":"2024-09-18T01:12:18.345763+0000","last_fullsized":"2024-09-18T01:12:18.345763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10101,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346067+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346067+0000","last_peered":"2024-09-18T01:12:18.346067+0000","last_clean":"2024-09-18T01:12:18.346067+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346067+0000","last_undegraded":"2024-09-18T01:12:18.346067+0000","last_fullsized":"2024-09-18T01:12:18.346067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7832,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345666+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345666+0000","last_peered":"2024-09-18T01:12:18.345666+0000","last_clean":"2024-09-18T01:12:18.345666+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345666+0000","last_undegraded":"2024-09-18T01:12:18.345666+0000","last_fullsized":"2024-09-18T01:12:18.345666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:45.894849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8589,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346101+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346101+0000","last_peered":"2024-09-18T01:12:18.346101+0000","last_clean":"2024-09-18T01:12:18.346101+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346101+0000","last_undegraded":"2024-09-18T01:12:18.346101+0000","last_fullsized":"2024-09-18T01:12:18.346101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4928,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345917+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345917+0000","last_peered":"2024-09-18T01:12:18.345917+0000","last_clean":"2024-09-18T01:12:18.345917+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345917+0000","last_undegraded":"2024-09-18T01:12:18.345917+0000","last_fullsized":"2024-09-18T01:12:18.345917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:34.188190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5424,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345409+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345409+0000","last_peered":"2024-09-18T01:12:18.345409+0000","last_clean":"2024-09-18T01:12:18.345409+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345409+0000","last_undegraded":"2024-09-18T01:12:18.345409+0000","last_fullsized":"2024-09-18T01:12:18.345409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:07.663503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71360,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300480,"statfs":{"total":96636764160,"available":96563691520,"internally_reserved":0,"allocated":4169728,"data_stored":2385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:21.433 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:12:21.434 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T01:12:21.727 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T01:12:21.742 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:12:21.742 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:22.018 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:22.019 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:22.031 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-18T01:12:20.779048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517832,"kb_used_data":29196,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456888,"statfs":{"total":773094113280,"available":772563853312,"internally_reserved":0,"allocated":29896704,"data_stored":15583005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001714"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4802,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346221+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346221+0000","last_peered":"2024-09-18T01:12:18.346221+0000","last_clean":"2024-09-18T01:12:18.346221+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346221+0000","last_undegraded":"2024-09-18T01:12:18.346221+0000","last_fullsized":"2024-09-18T01:12:18.346221+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13518,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345866+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:18.345866+0000","last_peered":"2024-09-18T01:12:18.345866+0000","last_clean":"2024-09-18T01:12:18.345866+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:18.345866+0000","last_undegraded":"2024-09-18T01:12:18.345866+0000","last_fullsized":"2024-09-18T01:12:18.345866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:35.072079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794718+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794718+0000","last_peered":"2024-09-18T01:12:17.794718+0000","last_clean":"2024-09-18T01:12:17.794718+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794718+0000","last_undegraded":"2024-09-18T01:12:17.794718+0000","last_fullsized":"2024-09-18T01:12:17.794718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4756,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345763+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345763+0000","last_peered":"2024-09-18T01:12:18.345763+0000","last_clean":"2024-09-18T01:12:18.345763+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345763+0000","last_undegraded":"2024-09-18T01:12:18.345763+0000","last_fullsized":"2024-09-18T01:12:18.345763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10101,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346067+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346067+0000","last_peered":"2024-09-18T01:12:18.346067+0000","last_clean":"2024-09-18T01:12:18.346067+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346067+0000","last_undegraded":"2024-09-18T01:12:18.346067+0000","last_fullsized":"2024-09-18T01:12:18.346067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7832,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345666+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345666+0000","last_peered":"2024-09-18T01:12:18.345666+0000","last_clean":"2024-09-18T01:12:18.345666+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345666+0000","last_undegraded":"2024-09-18T01:12:18.345666+0000","last_fullsized":"2024-09-18T01:12:18.345666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:45.894849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8589,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346101+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346101+0000","last_peered":"2024-09-18T01:12:18.346101+0000","last_clean":"2024-09-18T01:12:18.346101+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346101+0000","last_undegraded":"2024-09-18T01:12:18.346101+0000","last_fullsized":"2024-09-18T01:12:18.346101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4928,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345917+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345917+0000","last_peered":"2024-09-18T01:12:18.345917+0000","last_clean":"2024-09-18T01:12:18.345917+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345917+0000","last_undegraded":"2024-09-18T01:12:18.345917+0000","last_fullsized":"2024-09-18T01:12:18.345917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:34.188190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5424,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345409+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345409+0000","last_peered":"2024-09-18T01:12:18.345409+0000","last_clean":"2024-09-18T01:12:18.345409+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345409+0000","last_undegraded":"2024-09-18T01:12:18.345409+0000","last_fullsized":"2024-09-18T01:12:18.345409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:07.663503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71360,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300480,"statfs":{"total":96636764160,"available":96563691520,"internally_reserved":0,"allocated":4169728,"data_stored":2385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:22.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:22.310 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:22.322 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-18T01:12:20.779048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517832,"kb_used_data":29196,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456888,"statfs":{"total":773094113280,"available":772563853312,"internally_reserved":0,"allocated":29896704,"data_stored":15583005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001714"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4802,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346221+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346221+0000","last_peered":"2024-09-18T01:12:18.346221+0000","last_clean":"2024-09-18T01:12:18.346221+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346221+0000","last_undegraded":"2024-09-18T01:12:18.346221+0000","last_fullsized":"2024-09-18T01:12:18.346221+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13518,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345866+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:18.345866+0000","last_peered":"2024-09-18T01:12:18.345866+0000","last_clean":"2024-09-18T01:12:18.345866+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:18.345866+0000","last_undegraded":"2024-09-18T01:12:18.345866+0000","last_fullsized":"2024-09-18T01:12:18.345866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:35.072079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794718+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794718+0000","last_peered":"2024-09-18T01:12:17.794718+0000","last_clean":"2024-09-18T01:12:17.794718+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794718+0000","last_undegraded":"2024-09-18T01:12:17.794718+0000","last_fullsized":"2024-09-18T01:12:17.794718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4756,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345763+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345763+0000","last_peered":"2024-09-18T01:12:18.345763+0000","last_clean":"2024-09-18T01:12:18.345763+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345763+0000","last_undegraded":"2024-09-18T01:12:18.345763+0000","last_fullsized":"2024-09-18T01:12:18.345763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10101,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346067+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346067+0000","last_peered":"2024-09-18T01:12:18.346067+0000","last_clean":"2024-09-18T01:12:18.346067+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346067+0000","last_undegraded":"2024-09-18T01:12:18.346067+0000","last_fullsized":"2024-09-18T01:12:18.346067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7832,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345666+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345666+0000","last_peered":"2024-09-18T01:12:18.345666+0000","last_clean":"2024-09-18T01:12:18.345666+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345666+0000","last_undegraded":"2024-09-18T01:12:18.345666+0000","last_fullsized":"2024-09-18T01:12:18.345666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:45.894849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8589,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346101+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346101+0000","last_peered":"2024-09-18T01:12:18.346101+0000","last_clean":"2024-09-18T01:12:18.346101+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346101+0000","last_undegraded":"2024-09-18T01:12:18.346101+0000","last_fullsized":"2024-09-18T01:12:18.346101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4928,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345917+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345917+0000","last_peered":"2024-09-18T01:12:18.345917+0000","last_clean":"2024-09-18T01:12:18.345917+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345917+0000","last_undegraded":"2024-09-18T01:12:18.345917+0000","last_fullsized":"2024-09-18T01:12:18.345917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:34.188190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5424,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345409+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345409+0000","last_peered":"2024-09-18T01:12:18.345409+0000","last_clean":"2024-09-18T01:12:18.345409+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345409+0000","last_undegraded":"2024-09-18T01:12:18.345409+0000","last_fullsized":"2024-09-18T01:12:18.345409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:07.663503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71360,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300480,"statfs":{"total":96636764160,"available":96563691520,"internally_reserved":0,"allocated":4169728,"data_stored":2385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:22.326 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:12:22.326 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:22.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:22.603 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:22.614 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-18T01:12:20.779048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517832,"kb_used_data":29196,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456888,"statfs":{"total":773094113280,"available":772563853312,"internally_reserved":0,"allocated":29896704,"data_stored":15583005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001714"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4802,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346221+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346221+0000","last_peered":"2024-09-18T01:12:18.346221+0000","last_clean":"2024-09-18T01:12:18.346221+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346221+0000","last_undegraded":"2024-09-18T01:12:18.346221+0000","last_fullsized":"2024-09-18T01:12:18.346221+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13518,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345866+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:18.345866+0000","last_peered":"2024-09-18T01:12:18.345866+0000","last_clean":"2024-09-18T01:12:18.345866+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:18.345866+0000","last_undegraded":"2024-09-18T01:12:18.345866+0000","last_fullsized":"2024-09-18T01:12:18.345866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:35.072079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794718+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794718+0000","last_peered":"2024-09-18T01:12:17.794718+0000","last_clean":"2024-09-18T01:12:17.794718+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794718+0000","last_undegraded":"2024-09-18T01:12:17.794718+0000","last_fullsized":"2024-09-18T01:12:17.794718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4756,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345763+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345763+0000","last_peered":"2024-09-18T01:12:18.345763+0000","last_clean":"2024-09-18T01:12:18.345763+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345763+0000","last_undegraded":"2024-09-18T01:12:18.345763+0000","last_fullsized":"2024-09-18T01:12:18.345763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10101,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346067+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346067+0000","last_peered":"2024-09-18T01:12:18.346067+0000","last_clean":"2024-09-18T01:12:18.346067+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346067+0000","last_undegraded":"2024-09-18T01:12:18.346067+0000","last_fullsized":"2024-09-18T01:12:18.346067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7832,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345666+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345666+0000","last_peered":"2024-09-18T01:12:18.345666+0000","last_clean":"2024-09-18T01:12:18.345666+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345666+0000","last_undegraded":"2024-09-18T01:12:18.345666+0000","last_fullsized":"2024-09-18T01:12:18.345666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:45.894849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8589,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346101+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346101+0000","last_peered":"2024-09-18T01:12:18.346101+0000","last_clean":"2024-09-18T01:12:18.346101+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346101+0000","last_undegraded":"2024-09-18T01:12:18.346101+0000","last_fullsized":"2024-09-18T01:12:18.346101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4928,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345917+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345917+0000","last_peered":"2024-09-18T01:12:18.345917+0000","last_clean":"2024-09-18T01:12:18.345917+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345917+0000","last_undegraded":"2024-09-18T01:12:18.345917+0000","last_fullsized":"2024-09-18T01:12:18.345917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:34.188190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5424,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345409+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345409+0000","last_peered":"2024-09-18T01:12:18.345409+0000","last_clean":"2024-09-18T01:12:18.345409+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345409+0000","last_undegraded":"2024-09-18T01:12:18.345409+0000","last_fullsized":"2024-09-18T01:12:18.345409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:07.663503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71360,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300480,"statfs":{"total":96636764160,"available":96563691520,"internally_reserved":0,"allocated":4169728,"data_stored":2385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:22.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:12:22.897 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:22.898 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":296,"fsid":"e017e0c5-d4e1-4f77-8127-47c02525e65d","created":"2024-09-18T01:06:59.746320+0000","modified":"2024-09-18T01:12:17.679602+0000","last_up_change":"2024-09-18T01:11:58.860615+0000","last_in_change":"2024-09-18T01:10:43.698009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:07:10.752532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:07:17.736243+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:07:27.050688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"273","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":273,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13ff2fc9-b863-45cb-b111-62d2ff06dff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6825","nonce":1379550312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6827","nonce":1379550312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6831","nonce":1379550312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1379550312},{"type":"v1","addr":"172.21.15.120:6829","nonce":1379550312}]},"public_addr":"172.21.15.120:6825/1379550312","cluster_addr":"172.21.15.120:6827/1379550312","heartbeat_back_addr":"172.21.15.120:6831/1379550312","heartbeat_front_addr":"172.21.15.120:6829/1379550312","state":["exists","up"]},{"osd":1,"uuid":"01867b0a-aeb8-4b92-897b-f8176abf95a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":277,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6817","nonce":2892053774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6819","nonce":2892053774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6823","nonce":2892053774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2892053774},{"type":"v1","addr":"172.21.15.120:6821","nonce":2892053774}]},"public_addr":"172.21.15.120:6817/2892053774","cluster_addr":"172.21.15.120:6819/2892053774","heartbeat_back_addr":"172.21.15.120:6823/2892053774","heartbeat_front_addr":"172.21.15.120:6821/2892053774","state":["exists","up"]},{"osd":2,"uuid":"788e3030-dfc0-45b5-9c99-763dc9283283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6809","nonce":2679997431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6811","nonce":2679997431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6815","nonce":2679997431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":2679997431},{"type":"v1","addr":"172.21.15.120:6813","nonce":2679997431}]},"public_addr":"172.21.15.120:6809/2679997431","cluster_addr":"172.21.15.120:6811/2679997431","heartbeat_back_addr":"172.21.15.120:6815/2679997431","heartbeat_front_addr":"172.21.15.120:6813/2679997431","state":["exists","up"]},{"osd":3,"uuid":"9ecd2117-0d0b-4629-9e54-3578a68fa0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6801","nonce":712549013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6803","nonce":712549013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6807","nonce":712549013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":712549013},{"type":"v1","addr":"172.21.15.120:6805","nonce":712549013}]},"public_addr":"172.21.15.120:6801/712549013","cluster_addr":"172.21.15.120:6803/712549013","heartbeat_back_addr":"172.21.15.120:6807/712549013","heartbeat_front_addr":"172.21.15.120:6805/712549013","state":["exists","up"]},{"osd":4,"uuid":"3af2d4ee-bce6-49df-a10e-f7c367d9fcb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6818","nonce":3636718449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6822","nonce":3636718449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6830","nonce":3636718449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3636718449},{"type":"v1","addr":"172.21.15.162:6826","nonce":3636718449}]},"public_addr":"172.21.15.162:6818/3636718449","cluster_addr":"172.21.15.162:6822/3636718449","heartbeat_back_addr":"172.21.15.162:6830/3636718449","heartbeat_front_addr":"172.21.15.162:6826/3636718449","state":["exists","up"]},{"osd":5,"uuid":"fbac8f09-7655-4b5e-8bb5-74650aa3c2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6817","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6819","nonce":3011845137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6823","nonce":3011845137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6829","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6831","nonce":3011845137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6825","nonce":3011845137},{"type":"v1","addr":"172.21.15.162:6827","nonce":3011845137}]},"public_addr":"172.21.15.162:6819/3011845137","cluster_addr":"172.21.15.162:6823/3011845137","heartbeat_back_addr":"172.21.15.162:6831/3011845137","heartbeat_front_addr":"172.21.15.162:6827/3011845137","state":["exists","up"]},{"osd":6,"uuid":"1a98a211-cab7-414a-ad17-b06cede06bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":217,"up_from":277,"up_thru":281,"down_at":219,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6809","nonce":1013313713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6811","nonce":1013313713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6815","nonce":1013313713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1013313713},{"type":"v1","addr":"172.21.15.162:6813","nonce":1013313713}]},"public_addr":"172.21.15.162:6809/1013313713","cluster_addr":"172.21.15.162:6811/1013313713","heartbeat_back_addr":"172.21.15.162:6815/1013313713","heartbeat_front_addr":"172.21.15.162:6813/1013313713","state":["exists","up"]},{"osd":7,"uuid":"494f3743-c1f1-410f-b5f8-ce5927494495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6801","nonce":3795653175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6803","nonce":3795653175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6807","nonce":3795653175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3795653175},{"type":"v1","addr":"172.21.15.162:6805","nonce":3795653175}]},"public_addr":"172.21.15.162:6801/3795653175","cluster_addr":"172.21.15.162:6803/3795653175","heartbeat_back_addr":"172.21.15.162:6807/3795653175","heartbeat_front_addr":"172.21.15.162:6805/3795653175","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.476040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.253614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.779954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.553728+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.609031+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.157009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:09.118047+0000","dead_epoch":219},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:07:08.926014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7},{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:12:23.912 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T01:12:23.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:24.045 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:24.045 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:24.045 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:24.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T01:12:24.333 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T01:12:24.346 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T01:12:24.346 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:24.468 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:24.468 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:24.468 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:24.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T01:12:24.759 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T01:12:24.772 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T01:12:24.772 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:24.896 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:24.896 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:24.896 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:24.907 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T01:12:25.186 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T01:12:25.199 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T01:12:25.200 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:25.322 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:25.322 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:25.323 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:25.333 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T01:12:25.607 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T01:12:25.620 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T01:12:25.620 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:25.744 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:25.744 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:25.744 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:25.754 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T01:12:26.029 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T01:12:26.042 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T01:12:26.043 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:26.166 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:26.167 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:26.167 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:26.177 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T01:12:26.453 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T01:12:26.465 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T01:12:26.466 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:26.589 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:26.589 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-18T01:12:26.589 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:26.600 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T01:12:26.878 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T01:12:26.891 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T01:12:26.891 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:12:27.013 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-18T01:12:27.013 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:12:27.013 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-18T01:12:27.024 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T01:12:27.301 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T01:12:27.313 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:27.592 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:27.593 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:27.605 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-18T01:12:26.780427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26042,"ondisk_log_size":26042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517848,"kb_used_data":29208,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456872,"statfs":{"total":773094113280,"available":772563836928,"internally_reserved":0,"allocated":29908992,"data_stored":15588884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002512"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7440,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794615+0000","last_change":"2024-09-18T01:11:48.715458+0000","last_active":"2024-09-18T01:12:17.794615+0000","last_peered":"2024-09-18T01:12:17.794615+0000","last_clean":"2024-09-18T01:12:17.794615+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:17.794615+0000","last_undegraded":"2024-09-18T01:12:17.794615+0000","last_fullsized":"2024-09-18T01:12:17.794615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:52.297843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4803,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346264+0000","last_change":"2024-09-18T01:12:02.614962+0000","last_active":"2024-09-18T01:12:18.346264+0000","last_peered":"2024-09-18T01:12:18.346264+0000","last_clean":"2024-09-18T01:12:18.346264+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:18.346264+0000","last_undegraded":"2024-09-18T01:12:18.346264+0000","last_fullsized":"2024-09-18T01:12:18.346264+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4796,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346217+0000","last_change":"2024-09-18T01:11:48.731346+0000","last_active":"2024-09-18T01:12:18.346217+0000","last_peered":"2024-09-18T01:12:18.346217+0000","last_clean":"2024-09-18T01:12:18.346217+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:18.346217+0000","last_undegraded":"2024-09-18T01:12:18.346217+0000","last_fullsized":"2024-09-18T01:12:18.346217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:50.936850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"273'691","reported_seq":6109,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346315+0000","last_change":"2024-09-18T01:11:48.730987+0000","last_active":"2024-09-18T01:12:18.346315+0000","last_peered":"2024-09-18T01:12:18.346315+0000","last_clean":"2024-09-18T01:12:18.346315+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:18.346315+0000","last_undegraded":"2024-09-18T01:12:18.346315+0000","last_fullsized":"2024-09-18T01:12:18.346315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:57:01.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794477+0000","last_change":"2024-09-18T01:11:48.715712+0000","last_active":"2024-09-18T01:12:17.794477+0000","last_peered":"2024-09-18T01:12:17.794477+0000","last_clean":"2024-09-18T01:12:17.794477+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:17.794477+0000","last_undegraded":"2024-09-18T01:12:17.794477+0000","last_fullsized":"2024-09-18T01:12:17.794477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:00.082609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"273'522","reported_seq":7813,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686363+0000","last_change":"2024-09-18T01:11:48.720279+0000","last_active":"2024-09-18T01:12:17.686363+0000","last_peered":"2024-09-18T01:12:17.686363+0000","last_clean":"2024-09-18T01:12:17.686363+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:17.686363+0000","last_undegraded":"2024-09-18T01:12:17.686363+0000","last_fullsized":"2024-09-18T01:12:17.686363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:08.402771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9634,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346033+0000","last_change":"2024-09-18T01:12:02.601207+0000","last_active":"2024-09-18T01:12:18.346033+0000","last_peered":"2024-09-18T01:12:18.346033+0000","last_clean":"2024-09-18T01:12:18.346033+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:18.346033+0000","last_undegraded":"2024-09-18T01:12:18.346033+0000","last_fullsized":"2024-09-18T01:12:18.346033+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:55.928347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13519,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:20.774391+0000","last_change":"2024-09-18T01:11:48.739582+0000","last_active":"2024-09-18T01:12:20.774391+0000","last_peered":"2024-09-18T01:12:20.774391+0000","last_clean":"2024-09-18T01:12:20.774391+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:20.774391+0000","last_undegraded":"2024-09-18T01:12:20.774391+0000","last_fullsized":"2024-09-18T01:12:20.774391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7834,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794752+0000","last_change":"2024-09-18T01:11:48.715491+0000","last_active":"2024-09-18T01:12:17.794752+0000","last_peered":"2024-09-18T01:12:17.794752+0000","last_clean":"2024-09-18T01:12:17.794752+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:17.794752+0000","last_undegraded":"2024-09-18T01:12:17.794752+0000","last_fullsized":"2024-09-18T01:12:17.794752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:34.241911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4757,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345816+0000","last_change":"2024-09-18T01:11:48.730716+0000","last_active":"2024-09-18T01:12:18.345816+0000","last_peered":"2024-09-18T01:12:18.345816+0000","last_clean":"2024-09-18T01:12:18.345816+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:18.345816+0000","last_undegraded":"2024-09-18T01:12:18.345816+0000","last_fullsized":"2024-09-18T01:12:18.345816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:24.502638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10102,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346115+0000","last_change":"2024-09-18T01:11:48.707449+0000","last_active":"2024-09-18T01:12:18.346115+0000","last_peered":"2024-09-18T01:12:18.346115+0000","last_clean":"2024-09-18T01:12:18.346115+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:18.346115+0000","last_undegraded":"2024-09-18T01:12:18.346115+0000","last_fullsized":"2024-09-18T01:12:18.346115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:53.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6603,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686229+0000","last_change":"2024-09-18T01:11:48.720924+0000","last_active":"2024-09-18T01:12:17.686229+0000","last_peered":"2024-09-18T01:12:17.686229+0000","last_clean":"2024-09-18T01:12:17.686229+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:17.686229+0000","last_undegraded":"2024-09-18T01:12:17.686229+0000","last_fullsized":"2024-09-18T01:12:17.686229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:28.287137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":390,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345850+0000","last_change":"2024-09-18T01:11:59.594262+0000","last_active":"2024-09-18T01:12:18.345850+0000","last_peered":"2024-09-18T01:12:18.345850+0000","last_clean":"2024-09-18T01:12:18.345850+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:18.345850+0000","last_undegraded":"2024-09-18T01:12:18.345850+0000","last_fullsized":"2024-09-18T01:12:18.345850+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:08.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4984,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794700+0000","last_change":"2024-09-18T01:11:48.716817+0000","last_active":"2024-09-18T01:12:17.794700+0000","last_peered":"2024-09-18T01:12:17.794700+0000","last_clean":"2024-09-18T01:12:17.794700+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:17.794700+0000","last_undegraded":"2024-09-18T01:12:17.794700+0000","last_fullsized":"2024-09-18T01:12:17.794700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:15.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684930+0000","last_change":"2024-09-18T01:11:59.596560+0000","last_active":"2024-09-18T01:12:17.684930+0000","last_peered":"2024-09-18T01:12:17.684930+0000","last_clean":"2024-09-18T01:12:17.684930+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:17.684930+0000","last_undegraded":"2024-09-18T01:12:17.684930+0000","last_fullsized":"2024-09-18T01:12:17.684930+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:54.989969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7833,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345710+0000","last_change":"2024-09-18T01:11:48.739857+0000","last_active":"2024-09-18T01:12:18.345710+0000","last_peered":"2024-09-18T01:12:18.345710+0000","last_clean":"2024-09-18T01:12:18.345710+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:18.345710+0000","last_undegraded":"2024-09-18T01:12:18.345710+0000","last_fullsized":"2024-09-18T01:12:18.345710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346265+0000","last_change":"2024-09-18T01:07:21.867669+0000","last_active":"2024-09-18T01:12:18.346265+0000","last_peered":"2024-09-18T01:12:18.346265+0000","last_clean":"2024-09-18T01:12:18.346265+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:18.346265+0000","last_undegraded":"2024-09-18T01:12:18.346265+0000","last_fullsized":"2024-09-18T01:12:18.346265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:26.346964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7374,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684951+0000","last_change":"2024-09-18T01:11:48.708870+0000","last_active":"2024-09-18T01:12:17.684951+0000","last_peered":"2024-09-18T01:12:17.684951+0000","last_clean":"2024-09-18T01:12:17.684951+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:17.684951+0000","last_undegraded":"2024-09-18T01:12:17.684951+0000","last_fullsized":"2024-09-18T01:12:17.684951+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:29.829779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684939+0000","last_change":"2024-09-18T01:07:20.954917+0000","last_active":"2024-09-18T01:12:17.684939+0000","last_peered":"2024-09-18T01:12:17.684939+0000","last_clean":"2024-09-18T01:12:17.684939+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:17.684939+0000","last_undegraded":"2024-09-18T01:12:17.684939+0000","last_fullsized":"2024-09-18T01:12:17.684939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:27:00.410535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12917,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346330+0000","last_change":"2024-09-18T01:11:48.731287+0000","last_active":"2024-09-18T01:12:18.346330+0000","last_peered":"2024-09-18T01:12:18.346330+0000","last_clean":"2024-09-18T01:12:18.346330+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:18.346330+0000","last_undegraded":"2024-09-18T01:12:18.346330+0000","last_fullsized":"2024-09-18T01:12:18.346330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:46.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"273'739","reported_seq":9370,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346057+0000","last_change":"2024-09-18T01:12:03.625044+0000","last_active":"2024-09-18T01:12:18.346057+0000","last_peered":"2024-09-18T01:12:18.346057+0000","last_clean":"2024-09-18T01:12:18.346057+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:18.346057+0000","last_undegraded":"2024-09-18T01:12:18.346057+0000","last_fullsized":"2024-09-18T01:12:18.346057+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:20.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346199+0000","last_change":"2024-09-18T01:07:20.953060+0000","last_active":"2024-09-18T01:12:18.346199+0000","last_peered":"2024-09-18T01:12:18.346199+0000","last_clean":"2024-09-18T01:12:18.346199+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:18.346199+0000","last_undegraded":"2024-09-18T01:12:18.346199+0000","last_fullsized":"2024-09-18T01:12:18.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.752657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684812+0000","last_change":"2024-09-18T01:11:48.708305+0000","last_active":"2024-09-18T01:12:17.684812+0000","last_peered":"2024-09-18T01:12:17.684812+0000","last_clean":"2024-09-18T01:12:17.684812+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:17.684812+0000","last_undegraded":"2024-09-18T01:12:17.684812+0000","last_fullsized":"2024-09-18T01:12:17.684812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:35.584556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.684330+0000","last_change":"2024-09-18T01:10:44.648446+0000","last_active":"2024-09-18T01:12:17.684330+0000","last_peered":"2024-09-18T01:12:17.684330+0000","last_clean":"2024-09-18T01:12:17.684330+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:17.684330+0000","last_undegraded":"2024-09-18T01:12:17.684330+0000","last_fullsized":"2024-09-18T01:12:17.684330+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:43:50.253713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16267,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794554+0000","last_change":"2024-09-18T01:11:48.715207+0000","last_active":"2024-09-18T01:12:17.794554+0000","last_peered":"2024-09-18T01:12:17.794554+0000","last_clean":"2024-09-18T01:12:17.794554+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:17.794554+0000","last_undegraded":"2024-09-18T01:12:17.794554+0000","last_fullsized":"2024-09-18T01:12:17.794554+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:39.460539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346034+0000","last_change":"2024-09-18T01:07:21.867555+0000","last_active":"2024-09-18T01:12:18.346034+0000","last_peered":"2024-09-18T01:12:18.346034+0000","last_clean":"2024-09-18T01:12:18.346034+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:18.346034+0000","last_undegraded":"2024-09-18T01:12:18.346034+0000","last_fullsized":"2024-09-18T01:12:18.346034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:25.632740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8590,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346144+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.346144+0000","last_peered":"2024-09-18T01:12:18.346144+0000","last_clean":"2024-09-18T01:12:18.346144+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:18.346144+0000","last_undegraded":"2024-09-18T01:12:18.346144+0000","last_fullsized":"2024-09-18T01:12:18.346144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":602,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345886+0000","last_change":"2024-09-18T01:10:44.583585+0000","last_active":"2024-09-18T01:12:18.345886+0000","last_peered":"2024-09-18T01:12:18.345886+0000","last_clean":"2024-09-18T01:12:18.345886+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:18.345886+0000","last_undegraded":"2024-09-18T01:12:18.345886+0000","last_fullsized":"2024-09-18T01:12:18.345886+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:17.915428+0000","last_clean_scrub_stamp":"2024-09-18T01:07:17.915428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:41.883970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":719,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346175+0000","last_change":"2024-09-18T01:07:12.906867+0000","last_active":"2024-09-18T01:12:18.346175+0000","last_peered":"2024-09-18T01:12:18.346175+0000","last_clean":"2024-09-18T01:12:18.346175+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:18.346175+0000","last_undegraded":"2024-09-18T01:12:18.346175+0000","last_fullsized":"2024-09-18T01:12:18.346175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:10.880947+0000","last_clean_scrub_stamp":"2024-09-18T01:07:10.880947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:49.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4929,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345960+0000","last_change":"2024-09-18T01:11:48.740237+0000","last_active":"2024-09-18T01:12:18.345960+0000","last_peered":"2024-09-18T01:12:18.345960+0000","last_clean":"2024-09-18T01:12:18.345960+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:18.345960+0000","last_undegraded":"2024-09-18T01:12:18.345960+0000","last_fullsized":"2024-09-18T01:12:18.345960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12808,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.686370+0000","last_change":"2024-09-18T01:11:48.720093+0000","last_active":"2024-09-18T01:12:17.686370+0000","last_peered":"2024-09-18T01:12:17.686370+0000","last_clean":"2024-09-18T01:12:17.686370+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:17.686370+0000","last_undegraded":"2024-09-18T01:12:17.686370+0000","last_fullsized":"2024-09-18T01:12:17.686370+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:45.964340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10899,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345883+0000","last_change":"2024-09-18T01:11:59.706524+0000","last_active":"2024-09-18T01:12:18.345883+0000","last_peered":"2024-09-18T01:12:18.345883+0000","last_clean":"2024-09-18T01:12:18.345883+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:18.345883+0000","last_undegraded":"2024-09-18T01:12:18.345883+0000","last_fullsized":"2024-09-18T01:12:18.345883+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:12:05.615899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5425,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345465+0000","last_change":"2024-09-18T01:11:48.739635+0000","last_active":"2024-09-18T01:12:18.345465+0000","last_peered":"2024-09-18T01:12:18.345465+0000","last_clean":"2024-09-18T01:12:18.345465+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:18.345465+0000","last_undegraded":"2024-09-18T01:12:18.345465+0000","last_fullsized":"2024-09-18T01:12:18.345465+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7134,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346208+0000","last_change":"2024-09-18T01:11:48.731037+0000","last_active":"2024-09-18T01:12:18.346208+0000","last_peered":"2024-09-18T01:12:18.346208+0000","last_clean":"2024-09-18T01:12:18.346208+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:18.346208+0000","last_undegraded":"2024-09-18T01:12:18.346208+0000","last_fullsized":"2024-09-18T01:12:18.346208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:13.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14401,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346173+0000","last_change":"2024-09-18T01:11:48.708187+0000","last_active":"2024-09-18T01:12:18.346173+0000","last_peered":"2024-09-18T01:12:18.346173+0000","last_clean":"2024-09-18T01:12:18.346173+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:18.346173+0000","last_undegraded":"2024-09-18T01:12:18.346173+0000","last_fullsized":"2024-09-18T01:12:18.346173+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:02.245877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7983,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345911+0000","last_change":"2024-09-18T01:11:48.708131+0000","last_active":"2024-09-18T01:12:18.345911+0000","last_peered":"2024-09-18T01:12:18.345911+0000","last_clean":"2024-09-18T01:12:18.345911+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:18.345911+0000","last_undegraded":"2024-09-18T01:12:18.345911+0000","last_fullsized":"2024-09-18T01:12:18.345911+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:50.511019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6201,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345948+0000","last_change":"2024-09-18T01:11:48.708423+0000","last_active":"2024-09-18T01:12:18.345948+0000","last_peered":"2024-09-18T01:12:18.345948+0000","last_clean":"2024-09-18T01:12:18.345948+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:18.345948+0000","last_undegraded":"2024-09-18T01:12:18.345948+0000","last_fullsized":"2024-09-18T01:12:18.345948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.622090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4450,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345787+0000","last_change":"2024-09-18T01:11:59.852100+0000","last_active":"2024-09-18T01:12:18.345787+0000","last_peered":"2024-09-18T01:12:18.345787+0000","last_clean":"2024-09-18T01:12:18.345787+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:18.345787+0000","last_undegraded":"2024-09-18T01:12:18.345787+0000","last_fullsized":"2024-09-18T01:12:18.345787+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:15.169247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4990,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.346058+0000","last_change":"2024-09-18T01:11:48.707445+0000","last_active":"2024-09-18T01:12:18.346058+0000","last_peered":"2024-09-18T01:12:18.346058+0000","last_clean":"2024-09-18T01:12:18.346058+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:18.346058+0000","last_undegraded":"2024-09-18T01:12:18.346058+0000","last_fullsized":"2024-09-18T01:12:18.346058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:10.128270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7032,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:17.794563+0000","last_change":"2024-09-18T01:11:48.715163+0000","last_active":"2024-09-18T01:12:17.794563+0000","last_peered":"2024-09-18T01:12:17.794563+0000","last_clean":"2024-09-18T01:12:17.794563+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:17.794563+0000","last_undegraded":"2024-09-18T01:12:17.794563+0000","last_fullsized":"2024-09-18T01:12:17.794563+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:15.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6394,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:18.345743+0000","last_change":"2024-09-18T01:11:48.707939+0000","last_active":"2024-09-18T01:12:18.345743+0000","last_peered":"2024-09-18T01:12:18.345743+0000","last_clean":"2024-09-18T01:12:18.345743+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:18.345743+0000","last_undegraded":"2024-09-18T01:12:18.345743+0000","last_fullsized":"2024-09-18T01:12:18.345743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:07:27.960336+0000","last_clean_scrub_stamp":"2024-09-18T01:07:27.960336+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:51.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25922,"ondisk_log_size":25922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509509,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93584,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278256,"statfs":{"total":96636764160,"available":96540934144,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705940999,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509507,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66636,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305204,"statfs":{"total":96636764160,"available":96568528896,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509507,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509506,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56912,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71376,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300464,"statfs":{"total":96636764160,"available":96563675136,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74952,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296888,"statfs":{"total":96636764160,"available":96560013312,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70796,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301044,"statfs":{"total":96636764160,"available":96564269056,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:27.610 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.611 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.612 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.613 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T01:07:17.915428+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=17, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T01:07:10.880947+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=10, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.614 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T01:07:27.960336+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=7, tm_sec=27, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=12, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:12:27.615 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T01:12:47.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:12:47.908 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-18T01:12:47.909 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-18T01:12:47.920 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-18T01:12:46.785121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":266154,"num_read_kb":1262834,"num_write":36353,"num_write_kb":852403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26049,"ondisk_log_size":26049,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517784,"kb_used_data":29208,"kb_used_omap":23,"kb_used_meta":488424,"kb_avail":754456936,"statfs":{"total":773094113280,"available":772563902464,"internally_reserved":0,"allocated":29908992,"data_stored":15588884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23589,"internal_metadata":500147163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002871"},"pg_stats":[{"pgid":"3.1f","version":"273'572","reported_seq":7449,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.789403+0000","last_change":"2024-09-18T01:12:27.789426+0000","last_active":"2024-09-18T01:12:27.789403+0000","last_peered":"2024-09-18T01:12:27.789403+0000","last_clean":"2024-09-18T01:12:27.789403+0000","last_became_active":"2024-09-18T01:07:28.978455+0000","last_became_peered":"2024-09-18T01:07:28.978455+0000","last_unstale":"2024-09-18T01:12:27.789403+0000","last_undegraded":"2024-09-18T01:12:27.789403+0000","last_fullsized":"2024-09-18T01:12:27.789403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'572","last_scrub_stamp":"2024-09-18T01:12:27.789325+0000","last_deep_scrub":"273'572","last_deep_scrub_stamp":"2024-09-18T01:12:27.789325+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.789325+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:45.704707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00078194099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6482,"num_read_kb":39709,"num_write":665,"num_write_kb":34131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"273'775","reported_seq":4811,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:31.271587+0000","last_change":"2024-09-18T01:12:31.271630+0000","last_active":"2024-09-18T01:12:31.271587+0000","last_peered":"2024-09-18T01:12:31.271587+0000","last_clean":"2024-09-18T01:12:31.271587+0000","last_became_active":"2024-09-18T01:12:02.598143+0000","last_became_peered":"2024-09-18T01:12:02.598143+0000","last_unstale":"2024-09-18T01:12:31.271587+0000","last_undegraded":"2024-09-18T01:12:31.271587+0000","last_fullsized":"2024-09-18T01:12:31.271587+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'775","last_scrub_stamp":"2024-09-18T01:12:31.271480+0000","last_deep_scrub":"273'775","last_deep_scrub_stamp":"2024-09-18T01:12:31.271480+0000","last_clean_scrub_stamp":"2024-09-18T01:12:31.271480+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:53.508504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.015381865999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4986,"num_read_kb":34020,"num_write":1155,"num_write_kb":25809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"273'657","reported_seq":4806,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.443825+0000","last_change":"2024-09-18T01:12:28.443825+0000","last_active":"2024-09-18T01:12:28.443825+0000","last_peered":"2024-09-18T01:12:28.443825+0000","last_clean":"2024-09-18T01:12:28.443825+0000","last_became_active":"2024-09-18T01:07:28.972755+0000","last_became_peered":"2024-09-18T01:07:28.972755+0000","last_unstale":"2024-09-18T01:12:28.443825+0000","last_undegraded":"2024-09-18T01:12:28.443825+0000","last_fullsized":"2024-09-18T01:12:28.443825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'657","last_scrub_stamp":"2024-09-18T01:12:28.443703+0000","last_deep_scrub":"273'657","last_deep_scrub_stamp":"2024-09-18T01:12:28.443703+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.443703+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:23.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095910099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4392,"num_read_kb":36528,"num_write":976,"num_write_kb":39203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"296'692","reported_seq":6120,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:29.461685+0000","last_change":"2024-09-18T01:12:29.461685+0000","last_active":"2024-09-18T01:12:29.461685+0000","last_peered":"2024-09-18T01:12:29.461685+0000","last_clean":"2024-09-18T01:12:29.461685+0000","last_became_active":"2024-09-18T01:07:28.974504+0000","last_became_peered":"2024-09-18T01:07:28.974504+0000","last_unstale":"2024-09-18T01:12:29.461685+0000","last_undegraded":"2024-09-18T01:12:29.461685+0000","last_fullsized":"2024-09-18T01:12:29.461685+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'692","last_scrub_stamp":"2024-09-18T01:12:29.461566+0000","last_deep_scrub":"296'692","last_deep_scrub_stamp":"2024-09-18T01:12:29.461566+0000","last_clean_scrub_stamp":"2024-09-18T01:12:29.461566+0000","objects_scrubbed":1,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:26:18.204389+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00072072599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5516,"num_read_kb":39297,"num_write":922,"num_write_kb":35505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"273'1323","reported_seq":7823,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:24.751690+0000","last_change":"2024-09-18T01:12:24.751690+0000","last_active":"2024-09-18T01:12:24.751690+0000","last_peered":"2024-09-18T01:12:24.751690+0000","last_clean":"2024-09-18T01:12:24.751690+0000","last_became_active":"2024-09-18T01:07:28.974650+0000","last_became_peered":"2024-09-18T01:07:28.974650+0000","last_unstale":"2024-09-18T01:12:24.751690+0000","last_undegraded":"2024-09-18T01:12:24.751690+0000","last_fullsized":"2024-09-18T01:12:24.751690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'1323","last_scrub_stamp":"2024-09-18T01:12:24.751542+0000","last_deep_scrub":"273'1323","last_deep_scrub_stamp":"2024-09-18T01:12:24.751542+0000","last_clean_scrub_stamp":"2024-09-18T01:12:24.751542+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:42:18.192253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071293400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10328,"num_read_kb":42733,"num_write":1848,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"296'523","reported_seq":7824,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.861377+0000","last_change":"2024-09-18T01:12:27.861377+0000","last_active":"2024-09-18T01:12:27.861377+0000","last_peered":"2024-09-18T01:12:27.861377+0000","last_clean":"2024-09-18T01:12:27.861377+0000","last_became_active":"2024-09-18T01:10:18.583848+0000","last_became_peered":"2024-09-18T01:10:18.583848+0000","last_unstale":"2024-09-18T01:12:27.861377+0000","last_undegraded":"2024-09-18T01:12:27.861377+0000","last_fullsized":"2024-09-18T01:12:27.861377+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'523","last_scrub_stamp":"2024-09-18T01:12:27.861204+0000","last_deep_scrub":"296'523","last_deep_scrub_stamp":"2024-09-18T01:12:27.861204+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.861204+0000","objects_scrubbed":1,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:13:58.474337+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00050732900000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6585,"num_read_kb":38253,"num_write":647,"num_write_kb":25993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":22750810,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"273'704","reported_seq":9643,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:30.367946+0000","last_change":"2024-09-18T01:12:30.367980+0000","last_active":"2024-09-18T01:12:30.367946+0000","last_peered":"2024-09-18T01:12:30.367946+0000","last_clean":"2024-09-18T01:12:30.367946+0000","last_became_active":"2024-09-18T01:12:02.598416+0000","last_became_peered":"2024-09-18T01:12:02.598416+0000","last_unstale":"2024-09-18T01:12:30.367946+0000","last_undegraded":"2024-09-18T01:12:30.367946+0000","last_fullsized":"2024-09-18T01:12:30.367946+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'704","last_scrub_stamp":"2024-09-18T01:12:30.367807+0000","last_deep_scrub":"273'704","last_deep_scrub_stamp":"2024-09-18T01:12:30.367807+0000","last_clean_scrub_stamp":"2024-09-18T01:12:30.367807+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:07.217377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088878999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8986,"num_read_kb":43087,"num_write":788,"num_write_kb":27146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":6334546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"272'584","reported_seq":13527,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.362253+0000","last_change":"2024-09-18T01:12:27.362275+0000","last_active":"2024-09-18T01:12:27.362253+0000","last_peered":"2024-09-18T01:12:27.362253+0000","last_clean":"2024-09-18T01:12:27.362253+0000","last_became_active":"2024-09-18T01:07:28.973259+0000","last_became_peered":"2024-09-18T01:07:28.973259+0000","last_unstale":"2024-09-18T01:12:27.362253+0000","last_undegraded":"2024-09-18T01:12:27.362253+0000","last_fullsized":"2024-09-18T01:12:27.362253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'584","last_scrub_stamp":"2024-09-18T01:12:27.362170+0000","last_deep_scrub":"272'584","last_deep_scrub_stamp":"2024-09-18T01:12:27.362170+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.362170+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:33.454863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053455899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12954,"num_read_kb":57830,"num_write":679,"num_write_kb":24321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"273'1235","reported_seq":7843,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:25.763156+0000","last_change":"2024-09-18T01:12:25.763202+0000","last_active":"2024-09-18T01:12:25.763156+0000","last_peered":"2024-09-18T01:12:25.763156+0000","last_clean":"2024-09-18T01:12:25.763156+0000","last_became_active":"2024-09-18T01:07:28.975332+0000","last_became_peered":"2024-09-18T01:07:28.975332+0000","last_unstale":"2024-09-18T01:12:25.763156+0000","last_undegraded":"2024-09-18T01:12:25.763156+0000","last_fullsized":"2024-09-18T01:12:25.763156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'1235","last_scrub_stamp":"2024-09-18T01:12:25.763082+0000","last_deep_scrub":"273'1235","last_deep_scrub_stamp":"2024-09-18T01:12:25.763082+0000","last_clean_scrub_stamp":"2024-09-18T01:12:25.763082+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:39.158026+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00075772399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9152,"num_read_kb":41678,"num_write":1961,"num_write_kb":28920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"273'719","reported_seq":4766,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:32.487838+0000","last_change":"2024-09-18T01:12:32.487863+0000","last_active":"2024-09-18T01:12:32.487838+0000","last_peered":"2024-09-18T01:12:32.487838+0000","last_clean":"2024-09-18T01:12:32.487838+0000","last_became_active":"2024-09-18T01:07:28.978062+0000","last_became_peered":"2024-09-18T01:07:28.978062+0000","last_unstale":"2024-09-18T01:12:32.487838+0000","last_undegraded":"2024-09-18T01:12:32.487838+0000","last_fullsized":"2024-09-18T01:12:32.487838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'719","last_scrub_stamp":"2024-09-18T01:12:32.487743+0000","last_deep_scrub":"273'719","last_deep_scrub_stamp":"2024-09-18T01:12:32.487743+0000","last_clean_scrub_stamp":"2024-09-18T01:12:32.487743+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:35:45.930633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036399499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4597,"num_read_kb":22167,"num_write":978,"num_write_kb":19426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"273'976","reported_seq":10111,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:33.326663+0000","last_change":"2024-09-18T01:12:33.326724+0000","last_active":"2024-09-18T01:12:33.326663+0000","last_peered":"2024-09-18T01:12:33.326663+0000","last_clean":"2024-09-18T01:12:33.326663+0000","last_became_active":"2024-09-18T01:07:28.975992+0000","last_became_peered":"2024-09-18T01:07:28.975992+0000","last_unstale":"2024-09-18T01:12:33.326663+0000","last_undegraded":"2024-09-18T01:12:33.326663+0000","last_fullsized":"2024-09-18T01:12:33.326663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'976","last_scrub_stamp":"2024-09-18T01:12:33.326524+0000","last_deep_scrub":"273'976","last_deep_scrub_stamp":"2024-09-18T01:12:33.326524+0000","last_clean_scrub_stamp":"2024-09-18T01:12:33.326524+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:51:55.724416+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00051191699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10929,"num_read_kb":60639,"num_write":1340,"num_write_kb":30054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"273'700","reported_seq":6612,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:26.893093+0000","last_change":"2024-09-18T01:12:26.893141+0000","last_active":"2024-09-18T01:12:26.893093+0000","last_peered":"2024-09-18T01:12:26.893093+0000","last_clean":"2024-09-18T01:12:26.893093+0000","last_became_active":"2024-09-18T01:07:28.972837+0000","last_became_peered":"2024-09-18T01:07:28.972837+0000","last_unstale":"2024-09-18T01:12:26.893093+0000","last_undegraded":"2024-09-18T01:12:26.893093+0000","last_fullsized":"2024-09-18T01:12:26.893093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'700","last_scrub_stamp":"2024-09-18T01:12:26.892963+0000","last_deep_scrub":"273'700","last_deep_scrub_stamp":"2024-09-18T01:12:26.892963+0000","last_clean_scrub_stamp":"2024-09-18T01:12:26.892963+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:59.474305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313605,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6601,"num_read_kb":23624,"num_write":995,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":399,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.422923+0000","last_change":"2024-09-18T01:12:27.422951+0000","last_active":"2024-09-18T01:12:27.422923+0000","last_peered":"2024-09-18T01:12:27.422923+0000","last_clean":"2024-09-18T01:12:27.422923+0000","last_became_active":"2024-09-18T01:11:59.593926+0000","last_became_peered":"2024-09-18T01:11:59.593926+0000","last_unstale":"2024-09-18T01:12:27.422923+0000","last_undegraded":"2024-09-18T01:12:27.422923+0000","last_fullsized":"2024-09-18T01:12:27.422923+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:27.422825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:27.422825+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.422825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:25.559275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"273'805","reported_seq":4993,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:26.787139+0000","last_change":"2024-09-18T01:12:26.787160+0000","last_active":"2024-09-18T01:12:26.787139+0000","last_peered":"2024-09-18T01:12:26.787139+0000","last_clean":"2024-09-18T01:12:26.787139+0000","last_became_active":"2024-09-18T01:07:28.975634+0000","last_became_peered":"2024-09-18T01:07:28.975634+0000","last_unstale":"2024-09-18T01:12:26.787139+0000","last_undegraded":"2024-09-18T01:12:26.787139+0000","last_fullsized":"2024-09-18T01:12:26.787139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'805","last_scrub_stamp":"2024-09-18T01:12:26.787057+0000","last_deep_scrub":"273'805","last_deep_scrub_stamp":"2024-09-18T01:12:26.787057+0000","last_clean_scrub_stamp":"2024-09-18T01:12:26.787057+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:12.197426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017575480000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5184,"num_read_kb":27635,"num_write":1030,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":584,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.168453+0000","last_change":"2024-09-18T01:12:28.168472+0000","last_active":"2024-09-18T01:12:28.168453+0000","last_peered":"2024-09-18T01:12:28.168453+0000","last_clean":"2024-09-18T01:12:28.168453+0000","last_became_active":"2024-09-18T01:11:59.596256+0000","last_became_peered":"2024-09-18T01:11:59.596256+0000","last_unstale":"2024-09-18T01:12:28.168453+0000","last_undegraded":"2024-09-18T01:12:28.168453+0000","last_fullsized":"2024-09-18T01:12:28.168453+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:28.168381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:28.168381+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.168381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:24.593199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043773,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"272'495","reported_seq":7841,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:30.290438+0000","last_change":"2024-09-18T01:12:30.290478+0000","last_active":"2024-09-18T01:12:30.290438+0000","last_peered":"2024-09-18T01:12:30.290438+0000","last_clean":"2024-09-18T01:12:30.290438+0000","last_became_active":"2024-09-18T01:07:28.978868+0000","last_became_peered":"2024-09-18T01:07:28.978868+0000","last_unstale":"2024-09-18T01:12:30.290438+0000","last_undegraded":"2024-09-18T01:12:30.290438+0000","last_fullsized":"2024-09-18T01:12:30.290438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'495","last_scrub_stamp":"2024-09-18T01:12:30.290336+0000","last_deep_scrub":"272'495","last_deep_scrub_stamp":"2024-09-18T01:12:30.290336+0000","last_clean_scrub_stamp":"2024-09-18T01:12:30.290336+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:11.519354+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00072080500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6817,"num_read_kb":29999,"num_write":743,"num_write_kb":24542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":576,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:29.384059+0000","last_change":"2024-09-18T01:12:29.384116+0000","last_active":"2024-09-18T01:12:29.384059+0000","last_peered":"2024-09-18T01:12:29.384059+0000","last_clean":"2024-09-18T01:12:29.384059+0000","last_became_active":"2024-09-18T01:07:19.360004+0000","last_became_peered":"2024-09-18T01:07:19.360004+0000","last_unstale":"2024-09-18T01:12:29.384059+0000","last_undegraded":"2024-09-18T01:12:29.384059+0000","last_fullsized":"2024-09-18T01:12:29.384059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:29.383954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:29.383954+0000","last_clean_scrub_stamp":"2024-09-18T01:12:29.383954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:49:29.530138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048461899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"273'618","reported_seq":7383,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:26.183823+0000","last_change":"2024-09-18T01:12:26.183861+0000","last_active":"2024-09-18T01:12:26.183823+0000","last_peered":"2024-09-18T01:12:26.183823+0000","last_clean":"2024-09-18T01:12:26.183823+0000","last_became_active":"2024-09-18T01:10:47.725878+0000","last_became_peered":"2024-09-18T01:10:47.725878+0000","last_unstale":"2024-09-18T01:12:26.183823+0000","last_undegraded":"2024-09-18T01:12:26.183823+0000","last_fullsized":"2024-09-18T01:12:26.183823+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'618","last_scrub_stamp":"2024-09-18T01:12:26.183709+0000","last_deep_scrub":"273'618","last_deep_scrub_stamp":"2024-09-18T01:12:26.183709+0000","last_clean_scrub_stamp":"2024-09-18T01:12:26.183709+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:50.967528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069021800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6005,"num_read_kb":52760,"num_write":761,"num_write_kb":28706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":55738870,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":577,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:25.166876+0000","last_change":"2024-09-18T01:12:25.166895+0000","last_active":"2024-09-18T01:12:25.166876+0000","last_peered":"2024-09-18T01:12:25.166876+0000","last_clean":"2024-09-18T01:12:25.166876+0000","last_became_active":"2024-09-18T01:07:19.359708+0000","last_became_peered":"2024-09-18T01:07:19.359708+0000","last_unstale":"2024-09-18T01:12:25.166876+0000","last_undegraded":"2024-09-18T01:12:25.166876+0000","last_fullsized":"2024-09-18T01:12:25.166876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:25.166795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:25.166795+0000","last_clean_scrub_stamp":"2024-09-18T01:12:25.166795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:17.486212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00048296,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'983","reported_seq":12926,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:31.495925+0000","last_change":"2024-09-18T01:12:31.495952+0000","last_active":"2024-09-18T01:12:31.495925+0000","last_peered":"2024-09-18T01:12:31.495925+0000","last_clean":"2024-09-18T01:12:31.495925+0000","last_became_active":"2024-09-18T01:07:28.975664+0000","last_became_peered":"2024-09-18T01:07:28.975664+0000","last_unstale":"2024-09-18T01:12:31.495925+0000","last_undegraded":"2024-09-18T01:12:31.495925+0000","last_fullsized":"2024-09-18T01:12:31.495925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'983","last_scrub_stamp":"2024-09-18T01:12:31.495818+0000","last_deep_scrub":"273'983","last_deep_scrub_stamp":"2024-09-18T01:12:31.495818+0000","last_clean_scrub_stamp":"2024-09-18T01:12:31.495818+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:04:42.210619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090620399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14042,"num_read_kb":64892,"num_write":1474,"num_write_kb":38741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"296'740","reported_seq":9380,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:31.333356+0000","last_change":"2024-09-18T01:12:31.333401+0000","last_active":"2024-09-18T01:12:31.333356+0000","last_peered":"2024-09-18T01:12:31.333356+0000","last_clean":"2024-09-18T01:12:31.333356+0000","last_became_active":"2024-09-18T01:12:03.607546+0000","last_became_peered":"2024-09-18T01:12:03.607546+0000","last_unstale":"2024-09-18T01:12:31.333356+0000","last_undegraded":"2024-09-18T01:12:31.333356+0000","last_fullsized":"2024-09-18T01:12:31.333356+0000","mapping_epoch":281,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'740","last_scrub_stamp":"2024-09-18T01:12:31.333234+0000","last_deep_scrub":"296'740","last_deep_scrub_stamp":"2024-09-18T01:12:31.333234+0000","last_clean_scrub_stamp":"2024-09-18T01:12:31.333234+0000","objects_scrubbed":1,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:05.456468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015471166999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8556,"num_read_kb":43859,"num_write":934,"num_write_kb":24174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12364007,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"296'3","reported_seq":577,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.436934+0000","last_change":"2024-09-18T01:12:27.436966+0000","last_active":"2024-09-18T01:12:27.436934+0000","last_peered":"2024-09-18T01:12:27.436934+0000","last_clean":"2024-09-18T01:12:27.436934+0000","last_became_active":"2024-09-18T01:07:18.939067+0000","last_became_peered":"2024-09-18T01:07:18.939067+0000","last_unstale":"2024-09-18T01:12:27.436934+0000","last_undegraded":"2024-09-18T01:12:27.436934+0000","last_fullsized":"2024-09-18T01:12:27.436934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'3","last_scrub_stamp":"2024-09-18T01:12:27.436830+0000","last_deep_scrub":"296'3","last_deep_scrub_stamp":"2024-09-18T01:12:27.436830+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.436830+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:28:22.860005+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00096355200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"272'714","reported_seq":13652,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.165856+0000","last_change":"2024-09-18T01:12:27.165880+0000","last_active":"2024-09-18T01:12:27.165856+0000","last_peered":"2024-09-18T01:12:27.165856+0000","last_clean":"2024-09-18T01:12:27.165856+0000","last_became_active":"2024-09-18T01:10:47.728576+0000","last_became_peered":"2024-09-18T01:10:47.728576+0000","last_unstale":"2024-09-18T01:12:27.165856+0000","last_undegraded":"2024-09-18T01:12:27.165856+0000","last_fullsized":"2024-09-18T01:12:27.165856+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'714","last_scrub_stamp":"2024-09-18T01:12:27.165776+0000","last_deep_scrub":"272'714","last_deep_scrub_stamp":"2024-09-18T01:12:27.165776+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.165776+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:44:30.586493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049669600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12635,"num_read_kb":60629,"num_write":1069,"num_write_kb":28821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":54618423,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":193,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:25.844999+0000","last_change":"2024-09-18T01:12:25.845020+0000","last_active":"2024-09-18T01:12:25.844999+0000","last_peered":"2024-09-18T01:12:25.844999+0000","last_clean":"2024-09-18T01:12:25.844999+0000","last_became_active":"2024-09-18T01:10:44.648152+0000","last_became_peered":"2024-09-18T01:10:44.648152+0000","last_unstale":"2024-09-18T01:12:25.844999+0000","last_undegraded":"2024-09-18T01:12:25.844999+0000","last_fullsized":"2024-09-18T01:12:25.844999+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:25.844919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:25.844919+0000","last_clean_scrub_stamp":"2024-09-18T01:12:25.844919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:15.231398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"271'904","reported_seq":16276,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.757866+0000","last_change":"2024-09-18T01:12:28.757891+0000","last_active":"2024-09-18T01:12:28.757866+0000","last_peered":"2024-09-18T01:12:28.757866+0000","last_clean":"2024-09-18T01:12:28.757866+0000","last_became_active":"2024-09-18T01:09:57.578289+0000","last_became_peered":"2024-09-18T01:09:57.578289+0000","last_unstale":"2024-09-18T01:12:28.757866+0000","last_undegraded":"2024-09-18T01:12:28.757866+0000","last_fullsized":"2024-09-18T01:12:28.757866+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'904","last_scrub_stamp":"2024-09-18T01:12:28.757765+0000","last_deep_scrub":"271'904","last_deep_scrub_stamp":"2024-09-18T01:12:28.757765+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.757765+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:22:36.810001+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00065985399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16929,"num_read_kb":44875,"num_write":1038,"num_write_kb":34552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":16185146,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":576,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.371129+0000","last_change":"2024-09-18T01:12:28.371154+0000","last_active":"2024-09-18T01:12:28.371129+0000","last_peered":"2024-09-18T01:12:28.371129+0000","last_clean":"2024-09-18T01:12:28.371129+0000","last_became_active":"2024-09-18T01:07:18.939007+0000","last_became_peered":"2024-09-18T01:07:18.939007+0000","last_unstale":"2024-09-18T01:12:28.371129+0000","last_undegraded":"2024-09-18T01:12:28.371129+0000","last_fullsized":"2024-09-18T01:12:28.371129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:12:28.371038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:12:28.371038+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.371038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:10.287315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042442100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"272'532","reported_seq":8598,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.354779+0000","last_change":"2024-09-18T01:12:28.354801+0000","last_active":"2024-09-18T01:12:28.354779+0000","last_peered":"2024-09-18T01:12:28.354779+0000","last_clean":"2024-09-18T01:12:28.354779+0000","last_became_active":"2024-09-18T01:07:28.974749+0000","last_became_peered":"2024-09-18T01:07:28.974749+0000","last_unstale":"2024-09-18T01:12:28.354779+0000","last_undegraded":"2024-09-18T01:12:28.354779+0000","last_fullsized":"2024-09-18T01:12:28.354779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'532","last_scrub_stamp":"2024-09-18T01:12:28.354697+0000","last_deep_scrub":"272'532","last_deep_scrub_stamp":"2024-09-18T01:12:28.354697+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.354697+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:34:33.985992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086525399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7379,"num_read_kb":33307,"num_write":612,"num_write_kb":33623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"296'2","reported_seq":612,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:33.507267+0000","last_change":"2024-09-18T01:12:33.507300+0000","last_active":"2024-09-18T01:12:33.507267+0000","last_peered":"2024-09-18T01:12:33.507267+0000","last_clean":"2024-09-18T01:12:33.507267+0000","last_became_active":"2024-09-18T01:10:44.581846+0000","last_became_peered":"2024-09-18T01:10:44.581846+0000","last_unstale":"2024-09-18T01:12:33.507267+0000","last_undegraded":"2024-09-18T01:12:33.507267+0000","last_fullsized":"2024-09-18T01:12:33.507267+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'2","last_scrub_stamp":"2024-09-18T01:12:33.507177+0000","last_deep_scrub":"296'2","last_deep_scrub_stamp":"2024-09-18T01:12:33.507177+0000","last_clean_scrub_stamp":"2024-09-18T01:12:33.507177+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:16:02.539115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041381000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"296'119","reported_seq":730,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:27.377271+0000","last_change":"2024-09-18T01:12:27.377311+0000","last_active":"2024-09-18T01:12:27.377271+0000","last_peered":"2024-09-18T01:12:27.377271+0000","last_clean":"2024-09-18T01:12:27.377271+0000","last_became_active":"2024-09-18T01:07:12.905848+0000","last_became_peered":"2024-09-18T01:07:12.905848+0000","last_unstale":"2024-09-18T01:12:27.377271+0000","last_undegraded":"2024-09-18T01:12:27.377271+0000","last_fullsized":"2024-09-18T01:12:27.377271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'119","last_scrub_stamp":"2024-09-18T01:12:27.377170+0000","last_deep_scrub":"296'119","last_deep_scrub_stamp":"2024-09-18T01:12:27.377170+0000","last_clean_scrub_stamp":"2024-09-18T01:12:27.377170+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:22.701467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"272'1261","reported_seq":4937,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:26.401378+0000","last_change":"2024-09-18T01:12:26.401398+0000","last_active":"2024-09-18T01:12:26.401378+0000","last_peered":"2024-09-18T01:12:26.401378+0000","last_clean":"2024-09-18T01:12:26.401378+0000","last_became_active":"2024-09-18T01:07:28.973169+0000","last_became_peered":"2024-09-18T01:07:28.973169+0000","last_unstale":"2024-09-18T01:12:26.401378+0000","last_undegraded":"2024-09-18T01:12:26.401378+0000","last_fullsized":"2024-09-18T01:12:26.401378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'1261","last_scrub_stamp":"2024-09-18T01:12:26.401309+0000","last_deep_scrub":"272'1261","last_deep_scrub_stamp":"2024-09-18T01:12:26.401309+0000","last_clean_scrub_stamp":"2024-09-18T01:12:26.401309+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:26:45.811871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090120800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6668,"num_read_kb":21588,"num_write":2000,"num_write_kb":17124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"272'773","reported_seq":12817,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.824120+0000","last_change":"2024-09-18T01:12:28.824147+0000","last_active":"2024-09-18T01:12:28.824120+0000","last_peered":"2024-09-18T01:12:28.824120+0000","last_clean":"2024-09-18T01:12:28.824120+0000","last_became_active":"2024-09-18T01:10:47.728548+0000","last_became_peered":"2024-09-18T01:10:47.728548+0000","last_unstale":"2024-09-18T01:12:28.824120+0000","last_undegraded":"2024-09-18T01:12:28.824120+0000","last_fullsized":"2024-09-18T01:12:28.824120+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'773","last_scrub_stamp":"2024-09-18T01:12:28.824018+0000","last_deep_scrub":"272'773","last_deep_scrub_stamp":"2024-09-18T01:12:28.824018+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.824018+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:57.761704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044928200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12051,"num_read_kb":43895,"num_write":1069,"num_write_kb":21821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":56634894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"272'521","reported_seq":10908,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:28.385553+0000","last_change":"2024-09-18T01:12:28.385583+0000","last_active":"2024-09-18T01:12:28.385553+0000","last_peered":"2024-09-18T01:12:28.385553+0000","last_clean":"2024-09-18T01:12:28.385553+0000","last_became_active":"2024-09-18T01:11:59.596101+0000","last_became_peered":"2024-09-18T01:11:59.596101+0000","last_unstale":"2024-09-18T01:12:28.385553+0000","last_undegraded":"2024-09-18T01:12:28.385553+0000","last_fullsized":"2024-09-18T01:12:28.385553+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'521","last_scrub_stamp":"2024-09-18T01:12:28.385442+0000","last_deep_scrub":"272'521","last_deep_scrub_stamp":"2024-09-18T01:12:28.385442+0000","last_clean_scrub_stamp":"2024-09-18T01:12:28.385442+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:30.318440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0148334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10298,"num_read_kb":42924,"num_write":639,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"273'781","reported_seq":5433,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:29.306007+0000","last_change":"2024-09-18T01:12:29.306033+0000","last_active":"2024-09-18T01:12:29.306007+0000","last_peered":"2024-09-18T01:12:29.306007+0000","last_clean":"2024-09-18T01:12:29.306007+0000","last_became_active":"2024-09-18T01:07:28.976802+0000","last_became_peered":"2024-09-18T01:07:28.976802+0000","last_unstale":"2024-09-18T01:12:29.306007+0000","last_undegraded":"2024-09-18T01:12:29.306007+0000","last_fullsized":"2024-09-18T01:12:29.306007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'781","last_scrub_stamp":"2024-09-18T01:12:29.305915+0000","last_deep_scrub":"273'781","last_deep_scrub_stamp":"2024-09-18T01:12:29.305915+0000","last_clean_scrub_stamp":"2024-09-18T01:12:29.305915+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:27.499363+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00050196500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5307,"num_read_kb":33849,"num_write":1009,"num_write_kb":24422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"273'1037","reported_seq":7143,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:30.461848+0000","last_change":"2024-09-18T01:12:30.461876+0000","last_active":"2024-09-18T01:12:30.461848+0000","last_peered":"2024-09-18T01:12:30.461848+0000","last_clean":"2024-09-18T01:12:30.461848+0000","last_became_active":"2024-09-18T01:07:28.975418+0000","last_became_peered":"2024-09-18T01:07:28.975418+0000","last_unstale":"2024-09-18T01:12:30.461848+0000","last_undegraded":"2024-09-18T01:12:30.461848+0000","last_fullsized":"2024-09-18T01:12:30.461848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'1037","last_scrub_stamp":"2024-09-18T01:12:30.461708+0000","last_deep_scrub":"273'1037","last_deep_scrub_stamp":"2024-09-18T01:12:30.461708+0000","last_clean_scrub_stamp":"2024-09-18T01:12:30.461708+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:27.882359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066970799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7939,"num_read_kb":32686,"num_write":1686,"num_write_kb":22653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"272'720","reported_seq":14410,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:34.334920+0000","last_change":"2024-09-18T01:12:34.334946+0000","last_active":"2024-09-18T01:12:34.334920+0000","last_peered":"2024-09-18T01:12:34.334920+0000","last_clean":"2024-09-18T01:12:34.334920+0000","last_became_active":"2024-09-18T01:09:58.577813+0000","last_became_peered":"2024-09-18T01:09:58.577813+0000","last_unstale":"2024-09-18T01:12:34.334920+0000","last_undegraded":"2024-09-18T01:12:34.334920+0000","last_fullsized":"2024-09-18T01:12:34.334920+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'720","last_scrub_stamp":"2024-09-18T01:12:34.334811+0000","last_deep_scrub":"272'720","last_deep_scrub_stamp":"2024-09-18T01:12:34.334811+0000","last_clean_scrub_stamp":"2024-09-18T01:12:34.334811+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:51.787891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119797,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13929,"num_read_kb":53260,"num_write":961,"num_write_kb":27867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":21221107,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"272'1296","reported_seq":7992,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:35.333117+0000","last_change":"2024-09-18T01:12:35.333143+0000","last_active":"2024-09-18T01:12:35.333117+0000","last_peered":"2024-09-18T01:12:35.333117+0000","last_clean":"2024-09-18T01:12:35.333117+0000","last_became_active":"2024-09-18T01:09:58.578832+0000","last_became_peered":"2024-09-18T01:09:58.578832+0000","last_unstale":"2024-09-18T01:12:35.333117+0000","last_undegraded":"2024-09-18T01:12:35.333117+0000","last_fullsized":"2024-09-18T01:12:35.333117+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'1296","last_scrub_stamp":"2024-09-18T01:12:35.333013+0000","last_deep_scrub":"272'1296","last_deep_scrub_stamp":"2024-09-18T01:12:35.333013+0000","last_clean_scrub_stamp":"2024-09-18T01:12:35.333013+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:00.031342+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011494159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9391,"num_read_kb":56665,"num_write":2054,"num_write_kb":38820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":29792624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"272'666","reported_seq":6210,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:30.368350+0000","last_change":"2024-09-18T01:12:30.368392+0000","last_active":"2024-09-18T01:12:30.368350+0000","last_peered":"2024-09-18T01:12:30.368350+0000","last_clean":"2024-09-18T01:12:30.368350+0000","last_became_active":"2024-09-18T01:07:28.973004+0000","last_became_peered":"2024-09-18T01:07:28.973004+0000","last_unstale":"2024-09-18T01:12:30.368350+0000","last_undegraded":"2024-09-18T01:12:30.368350+0000","last_fullsized":"2024-09-18T01:12:30.368350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'666","last_scrub_stamp":"2024-09-18T01:12:30.368216+0000","last_deep_scrub":"272'666","last_deep_scrub_stamp":"2024-09-18T01:12:30.368216+0000","last_clean_scrub_stamp":"2024-09-18T01:12:30.368216+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:05.352681+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011001889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5782,"num_read_kb":31057,"num_write":919,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"272'580","reported_seq":4459,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:29.390246+0000","last_change":"2024-09-18T01:12:29.390284+0000","last_active":"2024-09-18T01:12:29.390246+0000","last_peered":"2024-09-18T01:12:29.390246+0000","last_clean":"2024-09-18T01:12:29.390246+0000","last_became_active":"2024-09-18T01:11:59.597205+0000","last_became_peered":"2024-09-18T01:11:59.597205+0000","last_unstale":"2024-09-18T01:12:29.390246+0000","last_undegraded":"2024-09-18T01:12:29.390246+0000","last_fullsized":"2024-09-18T01:12:29.390246+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"272'580","last_scrub_stamp":"2024-09-18T01:12:29.390100+0000","last_deep_scrub":"272'580","last_deep_scrub_stamp":"2024-09-18T01:12:29.390100+0000","last_clean_scrub_stamp":"2024-09-18T01:12:29.390100+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:14:56.202050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012586671000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3916,"num_read_kb":28645,"num_write":761,"num_write_kb":22810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"269'649","reported_seq":4999,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:31.322750+0000","last_change":"2024-09-18T01:12:31.322777+0000","last_active":"2024-09-18T01:12:31.322750+0000","last_peered":"2024-09-18T01:12:31.322750+0000","last_clean":"2024-09-18T01:12:31.322750+0000","last_became_active":"2024-09-18T01:07:28.974973+0000","last_became_peered":"2024-09-18T01:07:28.974973+0000","last_unstale":"2024-09-18T01:12:31.322750+0000","last_undegraded":"2024-09-18T01:12:31.322750+0000","last_fullsized":"2024-09-18T01:12:31.322750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'649","last_scrub_stamp":"2024-09-18T01:12:31.322655+0000","last_deep_scrub":"269'649","last_deep_scrub_stamp":"2024-09-18T01:12:31.322655+0000","last_clean_scrub_stamp":"2024-09-18T01:12:31.322655+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:06.052624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055708500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4984,"num_read_kb":27733,"num_write":840,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"273'1425","reported_seq":7041,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:29.754552+0000","last_change":"2024-09-18T01:12:29.754578+0000","last_active":"2024-09-18T01:12:29.754552+0000","last_peered":"2024-09-18T01:12:29.754552+0000","last_clean":"2024-09-18T01:12:29.754552+0000","last_became_active":"2024-09-18T01:10:00.570767+0000","last_became_peered":"2024-09-18T01:10:00.570767+0000","last_unstale":"2024-09-18T01:12:29.754552+0000","last_undegraded":"2024-09-18T01:12:29.754552+0000","last_fullsized":"2024-09-18T01:12:29.754552+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'1425","last_scrub_stamp":"2024-09-18T01:12:29.754443+0000","last_deep_scrub":"273'1425","last_deep_scrub_stamp":"2024-09-18T01:12:29.754443+0000","last_clean_scrub_stamp":"2024-09-18T01:12:29.754443+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:21:24.432750+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00061784000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9228,"num_read_kb":29760,"num_write":2298,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":6507798,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"271'965","reported_seq":6403,"reported_epoch":296,"state":"active+clean","last_fresh":"2024-09-18T01:12:32.324338+0000","last_change":"2024-09-18T01:12:32.324366+0000","last_active":"2024-09-18T01:12:32.324338+0000","last_peered":"2024-09-18T01:12:32.324338+0000","last_clean":"2024-09-18T01:12:32.324338+0000","last_became_active":"2024-09-18T01:07:28.973442+0000","last_became_peered":"2024-09-18T01:07:28.973442+0000","last_unstale":"2024-09-18T01:12:32.324338+0000","last_undegraded":"2024-09-18T01:12:32.324338+0000","last_fullsized":"2024-09-18T01:12:32.324338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'965","last_scrub_stamp":"2024-09-18T01:12:32.324234+0000","last_deep_scrub":"271'965","last_deep_scrub_stamp":"2024-09-18T01:12:32.324234+0000","last_clean_scrub_stamp":"2024-09-18T01:12:32.324234+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:23.563086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00097824999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7489,"num_read_kb":23150,"num_write":1341,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266037,"num_read_kb":1262733,"num_write":36192,"num_write_kb":849985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2792,"num_bytes_recovered":282148225,"num_keys_recovered":242,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},"log_size":25925,"ondisk_log_size":25925,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93572,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94278268,"statfs":{"total":96636764160,"available":96540946432,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":277,"seq":1189705941003,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":3504,"kb_used_omap":12,"kb_used_meta":32883,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12431,"internal_metadata":33673073},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66628,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305212,"statfs":{"total":96636764160,"available":96568537088,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47168,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324672,"statfs":{"total":96636764160,"available":96588464128,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56904,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314936,"statfs":{"total":96636764160,"available":96578494464,"internally_reserved":0,"allocated":3596288,"data_stored":1801047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509511,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71364,"kb_used_data":4084,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300476,"statfs":{"total":96636764160,"available":96563687424,"internally_reserved":0,"allocated":4182016,"data_stored":2391377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74948,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296892,"statfs":{"total":96636764160,"available":96560017408,"internally_reserved":0,"allocated":3592192,"data_stored":1801028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70784,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301056,"statfs":{"total":96636764160,"available":96564281344,"internally_reserved":0,"allocated":3588096,"data_stored":1801009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10837,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:12:47.923 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T01:12:48.217 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T01:12:48.217 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T01:12:48.218 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T01:12:48.218 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.258 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T01:12:48.258 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T01:12:48.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.302 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T01:12:48.302 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T01:12:48.302 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.343 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T01:12:48.343 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T01:12:48.344 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.382 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T01:12:48.383 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T01:12:48.383 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.425 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T01:12:48.425 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T01:12:48.425 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.466 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T01:12:48.466 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T01:12:48.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.534 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T01:12:48.534 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T01:12:48.534 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.609 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T01:12:48.609 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T01:12:48.609 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T01:12:48.610 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.633 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T01:12:48.634 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T01:12:48.634 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.678 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T01:12:48.679 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T01:12:48.679 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T01:12:48.679 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.690 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T01:12:48.690 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T01:12:48.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.742 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T01:12:48.743 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T01:12:48.743 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:12:48.756 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T01:12:48.756 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T01:12:48.757 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-18T01:12:48.824 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:12:48.824 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T01:12:49.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:12:49.089 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T01:12:49.194 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:12:49.195 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T01:12:49.258 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:12:49.258 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T01:12:49.310 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:12:49.310 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T01:12:49.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:12:49.459 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T01:12:49.560 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:12:49.560 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T01:12:49.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:12:49.615 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T01:12:49.686 INFO:tasks.ceph:Archiving mon data... 2024-09-18T01:12:49.687 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/data/mon.a.tgz 2024-09-18T01:12:49.687 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-18T01:12:49.703 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.yjcEbWCVzJ 2024-09-18T01:12:49.703 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.yjcEbWCVzJ 2024-09-18T01:12:49.890 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.yjcEbWCVzJ 2024-09-18T01:12:49.973 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.yjcEbWCVzJ is 527KB 2024-09-18T01:12:50.045 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.yjcEbWCVzJ 2024-09-18T01:12:50.059 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/data/mon.c.tgz 2024-09-18T01:12:50.060 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-18T01:12:50.114 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.UJvuSd3720 2024-09-18T01:12:50.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.UJvuSd3720 2024-09-18T01:12:50.328 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.UJvuSd3720 2024-09-18T01:12:50.410 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.UJvuSd3720 is 697KB 2024-09-18T01:12:50.511 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.UJvuSd3720 2024-09-18T01:12:50.526 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/data/mon.b.tgz 2024-09-18T01:12:50.526 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-18T01:12:50.545 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.d9RCyi0kRO 2024-09-18T01:12:50.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.d9RCyi0kRO 2024-09-18T01:12:50.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.d9RCyi0kRO 2024-09-18T01:12:50.858 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.d9RCyi0kRO is 697KB 2024-09-18T01:12:50.960 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.d9RCyi0kRO 2024-09-18T01:12:50.976 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T01:12:50.976 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:12:50.978 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:12:51.058 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:12:51.097 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:12:51.097 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T01:12:51.098 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi120/crash 2024-09-18T01:12:51.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:12:51.125 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi162/crash 2024-09-18T01:12:51.126 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:12:51.161 INFO:tasks.ceph:Compressing logs... 2024-09-18T01:12:51.162 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:12:51.170 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53990.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33106.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/objectstore_tool.53990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32946.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33026.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33106.log.gz 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32906.log 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32946.log.gz 2024-09-18T01:12:51.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33026.log.gz 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32906.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54919.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32906.log.gz 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/objectstore_tool.54919.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53101.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32850.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/objectstore_tool.53101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33146.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32850.log.gz 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33146.log.gz 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T01:12:51.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.log: 94.9% -- replaced with /var/log/ceph/objectstore_tool.53990.log.gz 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32986.log 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mgr.x.log: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32986.log.gz 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.54919.log.gz 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.53101.log.gz 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52190.log 2024-09-18T01:12:51.251 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-09-18T01:12:51.254 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/objectstore_tool.52190.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.52190.log.gz 2024-09-18T01:12:51.354 INFO:teuthology.orchestra.run.smithi162.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73580.log 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56295.log 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76464.log 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73580.log.gz 2024-09-18T01:12:51.386 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56295.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76464.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76422.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66880.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76422.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44609.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66880.log.gz 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52799.log 2024-09-18T01:12:51.387 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75811.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44609.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39244.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52799.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75036.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75811.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70726.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39244.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59547.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75036.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51767.log 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70726.log.gz 2024-09-18T01:12:51.388 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59547.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.74868.log -- replaced with /var/log/ceph/ceph-client.admin.59547.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68174.log 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51767.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74868.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.74868.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.61893.log 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68174.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54336.log 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61893.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70441.log 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-09-18T01:12:51.389 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33473.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54336.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58157.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70441.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.33473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33473.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69282.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58157.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65764.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77136.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69282.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76380.log 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65764.log.gz 2024-09-18T01:12:51.390 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.77136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77136.log.gz 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54059.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76380.log.gz 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gzgzip 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68566.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41013.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54059.log.gz 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68566.log.gz 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76128.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72016.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41013.log.gz 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73896.log 2024-09-18T01:12:51.391 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76128.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74532.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72016.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75596.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73896.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73957.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74532.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72631.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75596.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39246.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73957.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39878.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72631.log.gz 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60149.log 2024-09-18T01:12:51.392 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39246.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39878.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56860.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40156.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60149.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53020.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56860.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39237.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40156.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65326.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53020.log.gz 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68986.log 2024-09-18T01:12:51.393 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39237.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55611.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65326.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44389.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68986.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55611.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55611.log.gz -5 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39501.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62453.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44389.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39501.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39501.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62453.log.gz 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41337.log 2024-09-18T01:12:51.394 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41337.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41337.log.gz 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42678.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz --verbose 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.76254.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40726.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69850.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76254.log.gz 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62576.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40726.log.gz 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69850.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.69850.log.gz -- 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.63093.log 2024-09-18T01:12:51.395 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40282.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: 25.8% -- replaced with /var/log/ceph/ceph-client.admin.62576.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63093.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45434.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40282.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45434.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.45434.log.gz --verbose 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.52171.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74700.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52171.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67500.log 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-09-18T01:12:51.396 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74658.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74700.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.67500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70785.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67500.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65007.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74658.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74742.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70785.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66444.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65007.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74742.log.gz 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57619.log 2024-09-18T01:12:51.397 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66444.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62152.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57619.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47065.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73641.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62152.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47065.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72435.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63810.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73641.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54962.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72435.log.gz 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59604.log 2024-09-18T01:12:51.398 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63810.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54962.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54962.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.45813.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75255.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59604.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 0.0% 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64566.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75255.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75245.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42894.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64566.log.gz 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43101.log 2024-09-18T01:12:51.399 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75245.log.gz 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75256.log 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42894.log.gz 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43101.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.43101.log.gz /var/log/ceph/ceph-client.admin.60414.log 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75256.log.gz 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60414.log.gz 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46928.log 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56718.log 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74275.log 0.0% 2024-09-18T01:12:51.400 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46928.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56718.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.56718.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74910.log 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74275.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73780.log 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74616.log 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74910.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59666.log 0.0% 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73780.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68229.log 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74616.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59666.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42525.log -- replaced with /var/log/ceph/ceph-client.admin.59666.log.gz 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-09-18T01:12:51.401 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68229.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42525.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.75593.log 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67662.log 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40198.log 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75593.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.67662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67662.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52290.log 0.0% 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40198.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75488.log 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-09-18T01:12:51.402 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61577.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52290.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76800.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75488.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61577.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76800.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48225.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.38848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45856.log 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48225.log.gz 2024-09-18T01:12:51.403 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62314.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45856.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39022.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62314.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59029.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39022.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59029.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76674.log 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-09-18T01:12:51.404 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42587.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70374.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76674.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48384.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42587.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39581.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70374.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69995.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48384.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55294.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39581.log.gz 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69995.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.40240.log 2024-09-18T01:12:51.405 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69995.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61218.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55294.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40240.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40240.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.46333.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61218.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52757.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46333.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54458.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52757.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.52757.log.gz 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.76632.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42627.log 2024-09-18T01:12:51.406 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54458.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76632.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.60759.log -- replaced with /var/log/ceph/ceph-client.admin.76632.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55915.log 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42627.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70075.log 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60759.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40366.log 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55915.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55393.log 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70075.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51131.log 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40366.log.gz 2024-09-18T01:12:51.407 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55393.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56775.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55014.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51131.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65585.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56775.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55014.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63272.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65585.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75248.log 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63272.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.63272.log.gz 2024-09-18T01:12:51.408 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75248.log.gz 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41999.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40114.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73520.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40114.log.gz 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54715.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz -- 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.67837.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53826.log 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73520.log.gz 2024-09-18T01:12:51.409 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.67837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76842.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67837.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53826.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76506.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76842.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76506.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77010.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2024-09-18T01:12:51.410 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.77010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62495.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77010.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55732.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.50812.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48101.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62495.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50812.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48101.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-09-18T01:12:51.411 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66221.log 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39920.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.59828.log -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48869.log 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66221.log.gz 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49256.log 0.0% 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59828.log.gz 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56254.log 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48869.log.gz 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49256.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49256.log.gz /var/log/ceph/ceph-client.admin.46596.log 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50120.log 2024-09-18T01:12:51.412 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56254.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57579.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57498.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50120.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57579.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.57579.log.gz -- 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.75594.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71953.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57498.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41515.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75594.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69933.log 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71953.log.gz 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41515.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41515.log.gz -- 2024-09-18T01:12:51.413 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.41653.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44238.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69933.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41653.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53878.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74952.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44238.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39242.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53878.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76338.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74952.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39502.log 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39242.log.gz 2024-09-18T01:12:51.414 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53264.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76338.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69459.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39502.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53998.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53264.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58510.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69459.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53998.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60821.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58510.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66642.log 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-09-18T01:12:51.415 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60821.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56617.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66642.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76296.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56617.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74784.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75850.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76296.log.gz 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58075.log 2024-09-18T01:12:51.416 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74784.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55234.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75850.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51686.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58075.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71242.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55234.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64902.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42972.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71242.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54836.log 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64902.log.gz 2024-09-18T01:12:51.417 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62015.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42972.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54836.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63591.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62015.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60069.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58387.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63591.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76968.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60069.log.gz 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57739.log 2024-09-18T01:12:51.418 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58387.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69716.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76968.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57739.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69716.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44467.log 0.0% 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76758.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44467.log.gz 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77176.log 2024-09-18T01:12:51.419 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76758.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.77176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51951.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77176.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39876.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz /var/log/ceph/ceph-client.admin.47902.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72193.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51951.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72193.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2024-09-18T01:12:51.420 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61776.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64186.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71734.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61776.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64186.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65368.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71734.log: /var/log/ceph/ceph-client.admin.39500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71734.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46534.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65368.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.65368.log.gz 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76884.log 2024-09-18T01:12:51.421 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70911.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46534.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76884.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66145.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70911.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66145.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40324.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53660.log 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40324.log.gz 2024-09-18T01:12:51.422 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75078.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75260.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53660.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73460.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75078.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75260.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27174.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75258.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73460.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.tmp-client.admin.27174.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27174.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75944.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75258.log.gz 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2024-09-18T01:12:51.423 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75944.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54897.log 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58199.log 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60150.log 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54897.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58199.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58095.log 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46658.log 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60150.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58095.log.gz 2024-09-18T01:12:51.424 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75852.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46658.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76548.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75852.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76548.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60252.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48185.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60878.log 2024-09-18T01:12:51.425 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60252.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60647.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48185.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60878.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56816.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45377.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60647.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56816.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72867.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60717.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45377.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63875.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72867.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60717.log.gz 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41939.log 2024-09-18T01:12:51.426 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63875.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42263.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41939.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58791.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42263.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75496.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58791.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49214.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75496.log.gz 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74317.log 2024-09-18T01:12:51.427 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49214.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74448.log 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74317.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53321.log 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55490.log: /var/log/ceph/ceph-client.admin.74448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61617.log 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74448.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53321.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53321.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.42812.log 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61617.log.gz 2024-09-18T01:12:51.428 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72934.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72934.log.gz 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74406.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59364.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74406.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.74406.log.gz /var/log/ceph/ceph-client.admin.67215.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59364.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59364.log.gz 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.43883.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.67215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67215.log.gz 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-09-18T01:12:51.429 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.33058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43762.log: /var/log/ceph/ceph-client.admin.49296.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52841.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63314.log 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2024-09-18T01:12:51.430 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62834.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52841.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63314.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63314.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.70571.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62834.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71158.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70571.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73701.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44005.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71158.log: /var/log/ceph/ceph-client.admin.73701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71498.log 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73701.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71158.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44005.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44005.log.gz 2024-09-18T01:12:51.431 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.76926.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71498.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76212.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76926.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76212.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74233.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71200.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74233.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71200.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39919.log 2024-09-18T01:12:51.432 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39245.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39919.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38980.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39245.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.38980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38980.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38479.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.38479.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38479.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75935.log 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 2024-09-18T01:12:51.433 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73339.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75935.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73339.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39235.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53938.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39235.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50893.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53938.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69634.log 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip/var/log/ceph/ceph-client.admin.50893.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48647.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50893.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69634.log.gz 2024-09-18T01:12:51.434 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66541.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48647.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60151.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66541.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60151.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70973.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70973.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74574.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73184.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74574.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52599.log 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74574.log.gz 2024-09-18T01:12:51.435 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75595.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73184.log.gz 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52599.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65706.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46760.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75595.log.gz 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65706.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57816.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74364.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57816.log: /var/log/ceph/ceph-client.admin.46760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46760.log.gz 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57816.log.gz 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75257.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:12:51.436 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74364.log.gz 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75257.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49815.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58750.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.49815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49815.log.gz 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59927.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58750.log.gz 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50471.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59927.log.gz 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68670.log 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.437 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40481.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50471.log.gz 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68670.log.gz 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.0.40481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64944.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65904.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 95.5%/var/log/ceph/ceph-client.admin.64944.log: -- replaced with /var/log/ceph/ceph-client.0.40481.log.gz 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64944.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49028.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58018.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65904.log: /var/log/ceph/ceph-client.admin.49028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48707.log 2024-09-18T01:12:51.438 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65904.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49028.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58018.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58018.log.gz -5 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64847.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52089.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48707.log: /var/log/ceph/ceph-client.admin.64847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61259.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48707.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64847.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52089.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71098.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61259.log: /var/log/ceph/ceph-client.admin.51870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58531.log 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61259.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-09-18T01:12:51.439 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71098.log.gz 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42182.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58531.log: /var/log/ceph/ceph-client.admin.46697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58531.log.gz 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42182.log.gz 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58262.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39797.log: gzip/var/log/ceph/ceph-client.admin.58262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42080.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38904.log: -- /var/log/ceph/ceph-client.admin.63652.log 2024-09-18T01:12:51.440 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42080.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63652.log: -- /var/log/ceph/ceph-client.admin.39494.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63652.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52861.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72510.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39494.log: /var/log/ceph/ceph-client.admin.52861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62411.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52861.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72510.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.73400.log 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72510.log.gz 2024-09-18T01:12:51.441 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71376.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.62411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77094.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62411.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73400.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71376.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.77094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51211.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77094.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41817.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.64465.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51211.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63153.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64465.log.gz 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71870.log 2024-09-18T01:12:51.442 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.63153.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.49617.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63153.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71870.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49617.log: -- /var/log/ceph/ceph-client.admin.43800.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45773.log: -- /var/log/ceph/ceph-client.admin.62886.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41132.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62886.log: -- /var/log/ceph/ceph-client.admin.50075.log 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62886.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41132.log.gz 2024-09-18T01:12:51.443 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67878.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76716.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50075.log: /var/log/ceph/ceph-client.admin.67878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64804.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67878.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50075.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76716.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64804.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44912.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70849.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44912.log.gz 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-09-18T01:12:51.444 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70849.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55857.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74994.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55857.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74994.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71035.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39580.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71035.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41474.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39580.log.gz 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39064.log 2024-09-18T01:12:51.445 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41474.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64069.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39064.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57022.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64069.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54796.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57022.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73239.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54796.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65189.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73239.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65906.log 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65189.log.gz 2024-09-18T01:12:51.446 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.65906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65906.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77052.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.77052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77052.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70177.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50033.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72098.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.70177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70177.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76170.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.72098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72098.log.gz 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61121.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76170.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75812.log 2024-09-18T01:12:51.447 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76170.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.61121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44954.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61121.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60641.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75812.log: gzip/var/log/ceph/ceph-client.admin.44954.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76590.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44954.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0%/var/log/ceph/ceph-client.admin.60641.log: -- replaced with /var/log/ceph/ceph-client.admin.75812.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60641.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.76590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76590.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75940.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46154.log: gzip/var/log/ceph/ceph-client.admin.44771.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: /var/log/ceph/ceph-client.admin.69323.log 2024-09-18T01:12:51.448 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75940.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75940.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.69323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56534.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69323.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71714.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46174.log: /var/log/ceph/ceph-client.admin.56534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56534.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74826.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.71714.log: /var/log/ceph/ceph-client.admin.52556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71714.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59102.log 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-09-18T01:12:51.449 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.74826.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.74826.log.gz -5 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39582.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66900.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.59102.log: gzip/var/log/ceph/ceph-client.admin.39582.log: -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.75936.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39582.log.gz 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59102.log.gz 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.66900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66900.log.gz 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39241.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75936.log: /var/log/ceph/ceph-client.admin.55816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68607.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75936.log.gz 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.39241.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39241.log.gz -5 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-09-18T01:12:51.450 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68411.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68607.log: /var/log/ceph/ceph-client.admin.42381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33380.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68607.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68411.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58324.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.33380.log: /var/log/ceph/ceph-client.admin.53442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42728.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33380.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58324.log.gz 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74490.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38929.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42728.log: /var/log/ceph/ceph-client.admin.74490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2024-09-18T01:12:51.451 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42728.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74490.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.38929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38929.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59646.log 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75592.log 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50414.log: /var/log/ceph/ceph-client.admin.59646.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz -5 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41075.log 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59646.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75592.log: /var/log/ceph/ceph-client.admin.41075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73279.log 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41075.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75592.log.gz 2024-09-18T01:12:51.452 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47720.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.73279.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73279.log.gz 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68945.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47720.log.gz/var/log/ceph/ceph-client.admin.43444.log: 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75252.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.68945.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50615.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68945.log.gz 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.75252.log: /var/log/ceph/ceph-client.admin.50615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75252.log.gz 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50615.log.gz 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60515.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47663.log 2024-09-18T01:12:51.453 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.60515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60515.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51629.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75495.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47663.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51629.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64368.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52394.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.64368.log: /var/log/ceph/ceph-client.admin.75495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75495.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64368.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46885.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59208.log 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52394.log.gz 2024-09-18T01:12:51.454 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46885.log.gz 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58448.log 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57241.log 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.58448.log: /var/log/ceph/ceph-client.admin.59208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59208.log.gz 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58448.log.gz 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57241.log.gz 2024-09-18T01:12:51.455 INFO:teuthology.orchestra.run.smithi162.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T01:12:51.522 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T01:12:51.658 INFO:teuthology.orchestra.run.smithi120.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T01:12:51.844 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T01:12:52.822 INFO:teuthology.orchestra.run.smithi162.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T01:12:54.731 INFO:teuthology.orchestra.run.smithi120.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T01:12:55.123 INFO:teuthology.orchestra.run.smithi162.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T01:12:55.129 INFO:teuthology.orchestra.run.smithi162.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T01:12:56.125 INFO:teuthology.orchestra.run.smithi120.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T01:12:56.622 INFO:teuthology.orchestra.run.smithi162.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T01:12:56.624 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-18T01:12:56.624 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m5.404s 2024-09-18T01:12:56.624 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m14.485s 2024-09-18T01:12:56.624 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.628s 2024-09-18T01:12:57.014 INFO:teuthology.orchestra.run.smithi120.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T01:12:57.016 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-18T01:12:57.016 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m5.833s 2024-09-18T01:12:57.016 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m14.905s 2024-09-18T01:12:57.016 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.780s 2024-09-18T01:12:57.016 INFO:tasks.ceph:Archiving logs... 2024-09-18T01:12:57.017 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi120/log 2024-09-18T01:12:57.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:12:59.285 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi162/log 2024-09-18T01:12:59.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:13:01.057 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T01:13:01.066 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T01:13:01.067 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:13:01.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:13:01.146 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T01:13:01.155 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T01:13:01.155 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:13:01.158 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:13:01.172 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:13:01.173 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:13:01.177 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:13:01.178 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-18T01:13:01.178 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 -20us[ -124us] +/- 35ms 2024-09-18T01:13:01.178 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1482us[-1482us] +/- 46ms 2024-09-18T01:13:01.178 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +2245us[+2245us] +/- 74ms 2024-09-18T01:13:01.178 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 -28us[ -23us] +/- 35ms 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1485us[-1480us] +/- 46ms 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +2248us[+2253us] +/- 74ms 2024-09-18T01:13:01.179 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:13:01.179 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T01:13:01.201 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T01:13:01.201 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T01:13:01.236 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T01:13:01.331 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T01:13:01.412 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:13:01.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:13:01.443 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:13:01.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:13:01.473 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T01:13:01.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T01:13:01.548 INFO:teuthology.task.internal:Duration was 775.170208 seconds 2024-09-18T01:13:01.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T01:13:01.601 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T01:13:01.602 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:13:01.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:13:01.643 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:13:01.644 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:13:02.074 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T01:13:02.074 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-18T01:13:02.074 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:13:02.095 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-09-18T01:13:02.095 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:13:02.140 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T01:13:02.140 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:13:02.143 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:13:02.207 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T01:13:02.208 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:13:02.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:13:02.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T01:13:02.385 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T01:13:02.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:13:02.411 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:13:02.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T01:13:02.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:13:02.472 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:13:02.493 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-18T01:13:02.499 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-09-18T01:13:02.541 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:13:02.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:13:02.596 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:13:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:13:02.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T01:13:02.625 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T01:13:02.626 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi120 2024-09-18T01:13:02.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:13:02.676 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909695/remote/smithi162 2024-09-18T01:13:02.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:13:02.710 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T01:13:02.710 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:13:02.713 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:13:02.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T01:13:02.771 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T01:13:02.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T01:13:02.816 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T01:13:02.816 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:13:02.818 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:13:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:13 /home/ubuntu/cephtest 2024-09-18T01:13:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:13 /home/ubuntu/cephtest 2024-09-18T01:13:02.834 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T01:13:02.852 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T01:13:02.944 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} duration: 775.1702077388763 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-18T01:13:02.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T01:13:03.019 INFO:teuthology.run:pass